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PURPOSE 

This  guideline  is  a  general  reference  guide  for  preparing  for  and  implementing  a  Maintenance  Line  Operations 
Safety  Assessment  (M-LOSA)  program  or  Ramp  LOSA  (R-LOSA)  program.  The  intended  target  audiences  of  this 
guideline  are  LOSA  program  managers  and  members  of  the  implementation  team.  To  be  effective,  a  LOSA  program 
needs  to  address  each  individual  organizations  specific  needs  and  complement  existing  safety  programs.  However, 
the  following  are  core  components  that  ensure  the  integrity  of  any  LOSA  program: 

•  Peer-to-peer  observations  during  normal  operations; 

•  Confidential  and  non-punitive  data  collection; 

•  Voluntary  participation; 

•  Trusted  and  trained  observers; 

•  Joint  management/labor  sponsorship; 

•  Systematic  observation  instrument  based  on  Threat  and  Error  Management  (TEM)  model; 

•  Secure  data  collection  repository  and  mining; 

•  Feedback  of  results  to  the  workforce. 

Your  organization  may  want  to  consider  data-sharing  with  other  operators  to  enhance  the  power  of  data-derived 
knowledge.  To  maintain  consistency  and  comparability  in  LOSA  observation  data  and  to  allow  possible  industry¬ 
wide  information  sharing  in  the  future,  we  recommend  adopting  the  data  collection  forms,  training,  and  databases 
on  www.MRLOSA.com.  Over  a  course  of  three  years,  many  companies,  organizations,  and  associated  individuals 
devoted  numerous  hours  planning,  designing,  building,  testing,  and  refining  the  many  items  that  constitute  the  M- 
LOSA  and  R-LOSA  programs,  with  the  intent  of  the  widest  audience  use.  Further,  software  revision  and  support 
will  be  based  on  the  standardized  forms. 

Revision  Policy 

To  maintain  a  high  level  of  accuracy  and  timeliness  of  the  contents,  the  guideline  should  be  periodically  reviewed, 
revised,  and  updated. 
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•  Within  each  specific  revised  section  of  this  guideline,  including  appendices,  all  most-recent  revisions  to  the  language 
will  be  underlined.  Underlining  will  be  removed  from  any  revision  made  prior  to  the  most  recent. 


Implementation  Guideline  for  Maintenance  Line  Operations  Safety 
Assessment  (M-LOSA)  and  Ramp  LOSA  (R-LOSA)  Programs 


I.  Introduction 

Background 

Managing  risks  has  become  increasingly  important  in  modern  organizations.  The  initial  identification  and  interpretation 
of  hazards  are  some  of  the  most  challenging  aspects  of  risk  management,  since  many  hazards  remain  hidden,  unnoticed,  or 
misunderstood  for  long  periods  of  time  before  an  accident.  The  risks  associated  with  these  hazards  seem  obvious  after  an 
accident;  however,  the  early  signs  pointing  to  an  emerging  hazard  and  its  consequent  risk  are  often  extremely  weak  and 
ambiguous.  Three  sources  of  information  may  be  indicative  of  emerging  safety  risks:  (1)  reactive  sources  highlight  issues 
after  an  undesired  event  has  taken  place;  (2)  proactive  sources  look  for  precursors  to  undesired  events;  and  (3)  predictive 
sources  capture  system  performance  as  it  happens  in  real-time,  normal  operations  (Illson,  2006).  Line  Operations  Safety 
Assessment  (LOSA) 1  adopts  a  proactive  and  predictive  strategy  to  address  aviation  safety.  As  a  voluntary  safety  program, 
LOSA  collects  safety  data  during  normal  aviation  operations.  It  was  originally  designed  for  flight  deck  operations  and  has 
evolved  since  the  mid-nineties.  The  hazards  that  threaten  the  safety  of  flight  deck  operations  are  not  unique  to  that 
environment.  Similar  human  factors  problems  are  present  during  maintenance  and  ramp  operations. 

Project  development  background 

A  Maintenance  &  Ramp  Line  Operations  Safety  Assessment  (M-LOSA  &  R-LOSA)  project,  sponsored  by  the  Federal 
Aviation  Administration  (FAA),  was  launched  in  October  2008.  The  goal  was  to  capitalize  on  the  successes  of  flight  deck 
LOSA  and  extend  the  LOSA  methodology  to  aviation  maintenance  and  ramp  operations.  Though  using  a  similar 
methodology  employed  in  flight  deck  LOSA,  the  LOSA  programs2  would  have  to  be  reshaped  to  match  the  requirements 
of  maintenance  and  ramp  environments. 

We  consulted  numerous  airline  safety  representatives  who  were  engaged  in  current  maintenance  and  ramp  LOSA 
efforts.  These  personnel  brought  experience  from  thousands  of  LOSA  observations  from  around  the  world.  Review  of 
flight  deck  LOSA  and  existing  LOSA-themed  programs  in  maintenance  and  ramp  operations  informed  the  design  and 
prototyping  of  observation  data  collection  instruments  and  two  electronic  databases.  The  development  of  the  maintenance 
and  ramp  LOSA  procedures  and  training  took  an  immense  amount  of  effort  spanning  over  three  years.  After  initial 
development,  between  September  2009  and  November  2010,  beta  testing  allowed  input  from  over  100  maintenance 
technicians  and  ramp  personnel  at  five  U.S.  airports.  In  April  201 1,  a  maintenance  and  ramp  LOSA  trial  implementation 
was  conducted,  focusing  on  identifying  any  weaknesses  in  the  current  LOSA  programs  and  gaps  to  a  successful 
implementation.  Refer  to  www.MRLOSA.com  and  FAA  literature  review  report  (Ma  et  al.,  2011)  for  more  detailed 
information  about  the  M-LOSA  and  R-LOSA  projects. 

Threat  and  error  management 

Threat  and  Error  Management  (TEM)  provides  the  underlying  framework  for  LOSA  data  collection,  recognizing  that 
threats  and  errors  are  likely  to  occur  in  normal  operations.  The  TEM  model  allows  LOSA  observers  to  capture  the 
interaction  between  people  and  the  operational  context  by  recording  how  frontline  employees  manage  these  situations  to 
maintain  safety. 

There  are  three  basic  components  in  the  TEM  framework:  threats,  errors,  and  undesired  operational  states  (Maurino, 
2005).  Threats  are  considered  routine  events  that  must  be  managed  to  ensure  safety.  A  threat  is  any  condition  that 
increases  complexity  of  the  operations  that  demands  crew  attention  and  management  to  maintain  safety  margins. 

An  error  is  defined  as  a  crew  action  or  inaction  that  leads  to  a  deviation  from  organizational  intentions  or  expectations. 
Errors  ultimately  reduce  the  safety  margin  and  increase  the  probability  of  adverse  operational  events  on  the  ground  or 
during  flight.  Errors  normally  occur  when  threats  are  mismanaged.  However,  the  threat-error  linkage  is  not  necessarily 
straightforward,  and  it  may  not  always  be  possible  to  establish  a  one-to-one  mapping  between  threats  and  errors.  Errors 
can  be  spontaneous  without  direct  linkage  to  threats  (e.g.,  maintenance  technician  failing  to  give  a  required  callout  when 
there  is  no  distraction  in  the  hangar).  In  the  TEM  concept,  threats  and  errors  must  be  observable  (Maurino,  2005). 


1  LOSA  was  originally  an  acronym  for  “Line  Operations  Safety  Audit.”  To  better  promote  voluntary  participation  and  proactive  safety  culture, 
several  organizations  (e.g.,  International  Aviation  Transport  Association)  have  redefined  LOSA  as  “Line  Operations  Safety  Assessment.” 

2  LOSA  program(s)  in  this  guideline  refer  to  M-LOSA  and/or  R-LOSA  programs.  Flight  deck  LOSA  is  specified  as  such. 
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Error  outcomes  can  be  of  three  types.  Outcomes  of  errors  can  be  inconsequential  (i.e.,  no  effect  on  safety),  an  undesired 
operational  state  (a  risky  or  unsafe  condition  for  the  aircraft,  equipment,  and/or  personnel),  or  additional  error(s)  linked 
together  across  time.  Managing  an  undesired  operational  state  can  be  considered  the  last  opportunity  to  avoid  an  incident 
or  accident. 

Maintenance  technicians  and  ramp  employees  must  adopt  countermeasures  to  keep  threats,  errors,  and  undesired 
operational  states  from  affecting  safety  (Figure  1).  Threats  are  managed  by  applying  various  strategies  and  tactics.  Errors 
are  resolved  with  a  two-layered  approach  consisting  of  resist  and  resolve.  Resist  refers  to  the  variety  of  safety 
interventions  and  work  procedures  already  in  place  that  form  defenses  in  the  system  such  as  Standard  Operating 
Procedures  (SOPs),  checklists,  Quality  Control  (QC)  inspections,  and  automation.  Resolve  refers  to  the  maintenance 
technician  or  ramp  staff  realizing  that  they  had  made  an  error  and  then  correcting  it. 


Threat  &  Error  Management 

+  THREATS  + 


CONSEQUENCE 

Figure  1.  Threat  and  Error  Management  Model 

(Continental  Airlines,  2008.  Reprinted  with  permission.) 


The  TEM  model  is  aimed  at  understanding  error  management  (i.e.,  detection  and  response)  rather  than  solely  focusing 
on  error  causality  (i.e.,  causation  and  commission).  Regardless  of  the  error  type,  its  effect  on  safety  depends  on 
technicians’  and  ramp  employees’  detection  and  response  to  avoid  an  undesired  operational  state  and  prevent  a  potentially 
unsafe  outcome.  Using  the  powerful  yet  intuitive  TEM  framework,  LOSA  allows  safety  observers  to  detect  threats  and 
errors  that  technicians  and  ramp  employees  themselves  might  not  detect. 

Important  position  of  LOSA  in  SMS 

Safety  Management  System  (SMS)  is  becoming  a  worldwide  standard  throughout  the  aviation  industry.  The  essential 
idea  of  any  SMS  —  be  it  a  product/service  provider’s  SMS  or  the  SMS  of  the  regulator  responsible  for  safety  oversight  — 
is  to  provide  a  systematic  approach  to  achieving  acceptable  levels  of  safety  risk.  Serving  as  the  core  of  a  company’s  varied 
safety  efforts,  SMS  provides  an  organizational  framework  to  support  a  sound  safety  culture. 

SMS  is  comprised  of  four  functional  components,  which,  if  implemented  properly,  can  lead  to  a  safety  culture  within 
the  organization. 

•  Safety  Policy  —  Establishes  senior  management’s  commitment  to  continually  improve  safety;  defines  the 
methods,  processes,  and  organizational  structure  needed  to  meet  safety  goals. 

•  Safety  Risk  Management  (SRM)  —  Involves  the  identification  of  hazards,  assessment  of  the  hazards  for  risk,  and 
management  of  the  unacceptable  risks. 

•  Safety  Assurance  (SA)  —  Evaluates  the  effectiveness  of  the  SMS. 
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•  Safety  Promotion  —  Includes  training,  communication,  and  other  actions  to  train  all  levels  of  the  workforce  on 
their  role  in  the  SMS  and  to  motivate  them  to  willingly  carry  out  this  role. 

Popular  aviation  safety  programs  often  focus  on  the  “safety  risk  management”  aspect  of  SMS  (see  Figure  2)  and  can 
include  reactive,  proactive,  or  predictive  hazard  identification  programs.  Some  examples  include  (1)  voluntary  employee 
reporting  programs  such  as  the  Aviation  Safety  Action  Program  (ASAP)  and  the  Aviation  Safety  Reporting  System 
(ASRS),  which  are  proactive  programs,  (2)  data  analysis  programs  such  as  Flight  Operations  Quality  Assurance  (FOQA), 
which  is  a  predictive  program,  (3)  error  investigation  tools  such  as  the  Maintenance  Error  Decision  Aid/Ramp  Error 
Decision  Aid  (MED  A/RED  A),  which  are  reactive  programs,  and  (4)  system  assessment  programs  such  as  Continuing 
Analysis  and  Surveillance  System  (CASS),  which  is  a  proactive  program.  LOSA,  a  predictive  hazard  identification 
program,  complements  other  safety  programs  by  focusing  on  “safety  risk  management”  by  observing  the  system  during 
normal  operations.  M-LOSA  and  R-LOSA  are  now  available  to  organizations  as  predictive  hazard  identification  systems 
for  maintenance  and  ramp,  respectively.  LOSA  processes  are  critical  to  the  implementation  of  a  functional  SMS.  SMS 
also  provides  significant  business  benefits  such  as  better  operational  process  management  and,  consequently,  financial 
benefits. 


Safety  Management  System 


ASRS 


Safety  Promotion 


LOSA 


Figure  2.  Shift  of  Safety  Culture  in  Aviation  and  Various  Safety  Programs  in  SMS 


The  goal  of  SMS  is  to  (1)  encourage  the  accountable  manager  of  an  airline  to  take  responsibility  for  safety;  and  (2) 
understand  the  existing  hazards  and  maintain  the  risks  from  those  hazards  at  an  acceptable  level.  LOSA  helps  determine 
whether  best  practices  are  being  followed  and,  therefore,  supports  the  continuous  improvement  philosophy  behind  SMS. 

Purpose  of  LOSA 

LOSA  observers  are  subject  matter  experts  with  extensive  training  in  conducting  LOSA  observations.  They  record  data 
during  a  specific  observation  period.  LOSA  is  not  an  “audit”  program.  In  fact,  it  should  not  be  represented  or  used  as  an 
audit  program.  It  focuses  on  observing  normal  operations  by  peers  in  a  non-punitive  environment  to  identify  “at-risk” 
behaviors  to  implement  changes  to  get  employees  to  work  more  safely,  as  well  as  capture  information  on  effective 
countermeasures  currently  in  place.  LOSA  samples  activities  in  normal  operations  -  the  vast  majority  of  these  are  well- 
managed  and  successful  operations.  Confidential  data  collection  and  non-jeopardy  assurance  for  frontline  employees  are 
fundamental  to  the  process. 

Designed  to  be  a  project-based  safety  tool,  LOSA  brings  in  safety  information  derived  from  data.  LOSA  contributes  to 
continuous  quality  improvement  over  time.  As  a  behavior-based  safety  program,  LOSA  observes  human  behaviors 
(performing  specific  tasks).  There  is  no  intention  to  judge  whether  human  performance  is  good  or  bad,  efficient  or  not. 
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Using  a  medical  metaphor,  a  LOSA  is  a  patient’s  annual  physical  examination.  People  have  comprehensive  checkups  in 
the  hope  of  detecting  serious  health  issues  before  they  become  consequential.  A  set  of  diagnostic  measures,  such  as  blood 
pressure,  cholesterol,  and  liver  function,  can  flag  potential  health  concerns.  In  turn  the  physical  exam  could  suggest 
needed  changes  to  the  patient’s  current  lifestyle.  LOSA  is  built  upon  the  same  proactive  and  predictive  notion.  It  provides 
a  diagnostic  snapshot  of  strengths  and  weaknesses  that  an  aviation  organization  can  use  to  bolster  the  “health”  of  its  safety 
margins  and  prevent  degradation. 

The  development  and  success  of  LOSA  is  based  on  10  essential  characteristics: 

1 .  Peer-to-peer  observations  during  normal  operations, 

2.  Confidential  and  non-punitive  data  collection, 

3 .  V  oluntary  participation, 

4.  Trusted  and  trained  observers, 

5.  Joint  management/labor  sponsorship, 

6.  Systematic  observation  instrument  based  on  TEM  model, 

7.  Secure  data  collection  repository, 

8.  Data  verification  roundtables, 

9.  Data-derived  targets  for  enhancement,  and 

10.  Feedback  of  results  to  the  workforce. 

A  LOSA  program  offers  many  benefits.  To  highlight  a  few: 

•  Provides  field  data  to  support  and  improve  the  SMS, 

•  Identifies  strengths  and  weaknesses  of  normal  operations, 

•  Reduces  undesirable  events  and  operating  costs, 

•  Improves  efficiency, 

•  Improves  risk  management  practices  by  uncovering  at-risk  behaviors  (hazards),  and 

•  Sensitizes  LOSA  observers  and  workers  being  observed  to  constant  safe  work  practices. 

Overview  of  This  Guideline 

Including  the  introduction,  this  implementation  guideline  consists  of  1 1  sections  to  guide  the  LOSA 
implementation  process.  Each  section  covers  an  important  implementation  step  and  provides  guidelines,  tips,  key 
reminders,  and  additional  reading  for  that  particular  implementation  step. 

Practical  Implementation  Flow  Chart 

There  are  1 1  major  steps  in  implementing  a  LOSA  program  (see  Figure  3): 

1.  Obtain  senior  management’s  buy-in.  If  go-ahead  is  given,  then  take  the  following  steps: 

2.  Form  an  implementation  team; 

3.  Market  maintenance  and/or  ramp  LOSA  programs; 

4.  Integrate  with  existing  safety  programs/SMS; 

5.  Develop  LOSA  infrastructure,  including  three  parallel  activities: 

5.1  Adapt/customize  LOSA  database, 

5.2  Conduct  train-the-trainer  training, 

5.3  Establish  and  maintain  a  virtual  LOSA  website; 

6.  Adapt/customize  and  conduct  observer  training; 

7.  Collect  data; 

8.  Validate  data; 

9.  Populate  and  maintain  database; 

10.  Analyze  data  and  compile  a  report; 

11.  Provide  feedback  to  employees. 

If  considering  further  LOSA  effort  beyond  an  implementation,  there  are  more  steps  and  activities: 

12a.  Integrate  LOSA  data  and  data  from  other  safety  programs 
12b.  Analyze  Return  on  Investment  (ROI) 

12c.  Compile  success  stories  and  lessons  learned  and  add  them  to  the  training  program 
13.  Participate  in  industry  information  sharing  meeting 
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Figure  3.  A  Practical  Implementation  Flow  Chart  for  Maintenance  and  Ramp  LOSA 

Note:  A4A  -  Airlines  for  America  (formerly  known  as  Air  Transport  Association  of  America)  To  support  optimal 
view,  in  each  section  that  describes  an  implementation  step,  a  corresponding  segment  of  the  implementation  flow  chart 
will  be  called  out. 


FOR  ADDITIONAL  READING: 

A-  FAA.  (2006).  Advisory  Circular:  Line  Operations  Safety  Audits  (AC  120-90):  Federal  Aviation  Administration. 
^  FAA.  (2010).  FAA  Website  for  Safety  Management  System:  www.faa.gov/about/initiatives/sms 
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^  ICAO.  (2002).  Doc  9803  AN/761:  Line  Operations  Safety  Audit:  International  Civil  Aviation  Organization 
(ICAO). 

^  ICAO  Website  for  Integrated  Safety  Management  Section:  www2.icao.int/en/ism/default.aspx 
^  Illson,  J.  (2006,  November).  Integrated  Threat  Analysis .  Paper  presented  at  the  4th  ICAO-IATA  LOSA  &  TEM 
Conference,  Toulouse,  France. 

^  Ma,  J.,  Pedigo,  M.,  Blackwell,  L.,  Hackworth,  C.,  Holcomb,  K.,  &  Gildea,  K.  (201 1).  20  Years  of  the  Line 
Operations  Safety  Audit  (LOSA)  Program:  From  Flight  Operations  to  Maintenance  and  Ramp  Operations. 
(DOT/FAA/AM-1 1/15)  Washington,  DC:  Federal  Aviation  Administration,  Office  of  Aerospace  Medicine. 

^  Maurino,  D.  E.  (2005,  April).  Threat  and  Error  Management  (TEM).  Paper  presented  at  the  Canadian  Aviation 
Safety  Seminar  (CASS),  Vancouver,  BC,  Canada. 

4-  See  “Publications”  section  on  www.MRLOSA.com 
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II.  Preparation  Phase 


The  preparation  phase  consists  of  five  steps:  Assessing  Readiness,  Obtaining  Senior  Management’s  Buy-in,  Forming  a 
LOSA  Implementation  Team,  Marketing,  and  Integrating  With  Existing  Safety  Programs/SMS. 


Assess  Readiness 

Before  implementing  a  LOSA  program,  an  organization  should  first  assess  its  own  readiness  for  LOSA  by  using  the 
following  two  checklists. 

Must-have  checklist: 

□  Senior  management’s  committed  support  for  the  LOSA  effort  (including  financial  commitment). 

□  Labor  union  and/or  employee  groups’  acceptance  and  participation  for  the  LOSA  effort. 

□  Labor  force’s  acceptance  and  participation  for  the  LOSA  effort. 

Nice-to-have  checklist: 

□  Safety  personnel  are  familiar  with  the  LOSA  concept. 

a.  There  is  a  flight  or  other  LOSA  program  in  place. 

b.  There  is  a  risk  management  program  or  system  in  place. 

□  Non-punitive  safety  programs  such  as  ASAP,  ASRS  in  place. 

□  At  least  one  formal  safety  data  collection  and  trending  program  in  place. 

□  A  human  factors  program  in  place. 

□  Organizational  support  for  Just  Culture. 

□  Familiar  with  SMS 

a.  Has  your  organization  developed  a  SMS? 

b.  How  mature  is  your  SMS?  (e.g.,  Maturity  Levels  0-4, 

www.  faa.  gov/  about/ initiatives/sms/pilot_proj  ects/afs_process/index.  cfm?print=go) . 

The  above  checklist  items  and  questions  assess  the  different  aspects  of  an  internal  safety  environment  and  the 
likelihood  for  LOSA  programs  to  grow  successfully.  If  you  do  not  have  all  three  must-have  items,  your  organization  is  not 
ready  to  implement  a  LOSA  program.  Even  if  you  have  all  the  must-have  items,  but  none  or  only  one  or  two  of  the  nice- 
to-have  items,  your  organization  is  unlikely  to  be  truly  ready  for  LOSA  program  to  thrive.  Address  any  issues  you  identify 
first,  and  then  come  back  to  prepare  for  a  LOSA  implementation. 


Step  1:  Obtain  Senior  Management  Buy-In 

For  a  LOSA  program  to  succeed  senior  and  upper  management  must  be  fully  sold  on  the  safety  and  financial  benefits 
of  the  LOSA  programs  (see  Figure  4).  They  must  consistently  offer  their  committed  support.  The  LOSA  marketing 
presentation  for  the  highest  level  executives  and  management  explains  the  background  of  the  LOSA  project,  various 
players  involved,  LOSA  products,  implementation  process,  maintenance  or  ramp  human  factors  at  airlines  or  ground 
operators  worldwide,  and  so  on. 
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Figure  4.  Begin  the  LOSA  Implementation  by  Obtaining 
Senior  Management's  Buy-In 

It  is  critical  to  obtain  senior  management’s  commitment  to  share  fact-based  data  on  LOSA  effectiveness  with  the 
industry  as  the  LOSA  programs  grow.  Positive  outcomes  speak  for  themselves.  Fact-based  data  are  powerful  at  helping 
move  M-LOSA  &  R-LOSA  toward  broad  implementation  and  acceptance.  Only  senior  management  will  have  the 
corporate-wide  positive  influence  regarding  sharing  de-identified  and  aggregated  fact-based  data,  on  departments  such  as 
Legal,  Public  Relations,  and  Corporate  Communication.  Since  these  departments  influence  the  timely  publication  of 
information,  it  is  critical  that  senior  management  not  let  them  become  a  roadblock  to  success  and  to  safety. 

The  LOSA  marketing  presentation  or  the  C-suite  and  other  management  takes  approximately  45  minutes  to  deliver, 
including  questions  and  answers.  It  is  highly  recommended  to  deliver  it  in  a  face-to-face  format  to  the  level  of  senior 
management  that  has  the  ability  to  commit  the  required  resources  to  implement  the  program.  We  have  had  exceptional 
success  promoting  LOSA  programs  to  management  teams,  often  with  senior  pilot  representation,  who  have  had  success 
with  LOSA  on  the  flight  deck.  If  your  organization  has  experience  with  LOSA  on  the  flight  deck,  then  it  is  important  to 
bring  those  advocates  to  the  executive  meeting. 

Step  2:  Form  an  Implementation  Team 

It  is  critical  to  form  a  LOSA  implementation  team  to  develop  a  plan  for  the  LOSA  implementation  (see  Figure  5).  The 
plan  must  have  specific  goals,  objectives,  scope  of  the  implementation,  timeline,  and  success  measures.  The 
implementation  team  should  champion  the  marketing  effort,  training,  and  forming  partnerships.  It  will  oversee  the 
implementation  process  and  also  monitor  and  evaluate  the  team’s  success  toward  achieving  the  plan’s  goals  and 
objectives. 


Figure  5.  Forming  a  LOSA  Implementation  Team 


The  implementation  team  must  clearly  understand  the  level  of  commitment  that  an  organization  has  for  LOSA.  The 
team  may  want  to  form  various  working  groups  as  needed.  The  LOSA  implementation  team  (8-10  key  members, 
depending  on  the  scale  of  initial  implementation)  should  include  both  internal  and  external  members: 

•  Internal  members  should  represent  at  least  the  following  departments  and  functions:  safety,  operations,  training, 
employee  representatives,  senior  management,  LOSA  program  manager,  and  data  analysts. 

•  All  team  members  should  be  willing  participants  and  generally  support  safety  and/or  continuous  improvements. 
Beware  of  members  who  were  “volunteered.” 

•  External  members:  industry  representatives. 

The  LOSA  program  manager  should  assume  the  leadership  role  for  the  implementation  team.  External  members  on  the 
implementation  team  support  and  advise  on  activities  such  as  use  of  LOSA  observation  forms,  observer  selection  and 
training,  and  data  collection  and  analysis. 

The  operations,  quality  assurance  (QA),  and  training  departments  typically  know  first-hand  what  is  and  is  not  working 
well  in  their  organizations.  These  departments  often  have  specific  areas  of  concern  upon  which  they  would  like  the  LOSA 
observations  to  focus.  The  operating  departments  need  to  be  included  in  the  implementation  process.  If  these  departments 
do  not  support  LOSA,  then  there  may  be  resistance  to  the  findings  and  any  subsequent  corrective  actions. 

The  importance  of  having  the  labor  union  and/or  employee  groups  involved  with  and  supporting  the  LOSA  cannot  be 
overstated.  If  the  frontline  employees  are  convinced  that  their  union  supports  LOSA,  they  will  be  more  willing  to  accept 
the  presence  of  observers.  Additionally,  if  the  employees  believe  this  process  is  beneficial  to  their  daily  work  effort  and 
safety,  they  will  be  forthcoming  and  candid  with  their  views  and  safety  concerns.  On  the  other  hand,  if  the  technicians  or 
ramp  employees  view  LOSA  as  a  management  tool  to  “spy  on  their  performance”  and  they  respond  with  “angel” 
performance  rather  than  typical  performance,  then  the  results  will  not  be  fruitful. 

Main  tasks  during  the  preparation  stage 

The  LOSA  implementation  team  is  responsible  for  all  steps  outlined  in  the  practical  LOSA  implementation  flow  chart 
(Figure  3).  In  the  preparation  stage,  the  team  will  engage  in  the  following  tasks. 

(1)  Gather  Information  and  LOSA  Resources  From  Other  Organizations  and  Industry  Groups.  Before 
conducting  a  LOSA  for  the  first  time,  the  implementation  team  and  the  program  manager  should  review  all  materials 
provided  on  www.MRLOSA.com.  It  is  helpful  to  seek  out  information  from  other  organizations  that  have  already 
conducted  a  LOSA.  Other  airlines  and  ground  operators  may  be  able  to  share  observer  selection  and  training  techniques, 
customized  observation  forms,  scheduling  tips,  and  other  logistical  aids. 

(2)  Publicize  LOSA  Within  Your  Organization  and  Send  a  Letter  to  the  Frontline  Employees.  A  first  task  is  to 
publicize  LOSA  via  company  publications  and  communication  venues  to  build  employee  awareness  and  acceptance.  Next, 
the  implementation  team  should  organize  and  distribute  a  letter  to  all  technicians  or  ramp  employees  explaining  the 
purpose  of  the  LOSA  that  clearly  describes  the  goals  of  the  process.  This  letter  specifies  the  purpose  of  the  observations, 
the  fact  that  all  observations  are  of  a  non-jeopardy  nature,  and  that  all  data  are  de-identified  and  will  be  kept  strictly 
anonymous.  The  letter  should  be  signed  by  the  highest  level  of  management  within  maintenance  or  ground  operations; 
with  the  endorsement  of  other  relevant  personnel  such  as  supervisors’  and  employee  groups’  representatives.  The  letter  of 
announcement  should  precede  the  observations  by  at  least  one  month,  with  a  follow-up  reminder  one  week  before  starting 
observations.  LOSA  observers  should  have  copies  of  the  signed  letter  to  show  technicians  or  ramp  employees  in  case 
questions  arise.  In  addition  to  the  letter,  it  is  important  for  the  implementation  team  to  organize  all  employees  to  attend 
LOSA  awareness  training,  which  will  be  discussed  in  detail  in  the  “LOSA  Infrastructure  Development  Phase”  of  this 
guideline. 

(3)  Decide  on  the  Focus  of  the  LOSA.  The  implementation  team  should  develop  an  implementation  strategy.  One 
option  is  to  carry  out  LOSA  observations  on  a  broad  sample  of  the  entire  operation  —  this  would  be  an  effective  strategy 
for  a  first  LOSA.  Alternatively,  the  LOSA  implementation  team  can  focus  on  specific  areas  that  have  been  identified  by 
other  data  sources,  such  as  ASAP.  This  approach  would  schedule  LOSA  observations  on  particular  types  of  tasks,  on 
particular  aircraft  types,  or  in  particular  locations  that  have  been  identified  as  problematic.  The  implementation  team  can 
also  focus  a  LOSA  on  a  new  aircraft  type  in  the  fleet  or  other  recent  organizational  changes.  If  a  phased  approach  is 
adopted,  such  as  location  by  location,  the  operator  should  tailor  the  communication  plan  accordingly. 

(4)  Decide  on  the  Size  of  Implementation.  Always  start  with  a  trial  LOSA  implementation,  focusing  on  a  particular 
operation  or  region,  for  example,  a  single  station  instead  of  a  hub.  Depending  on  the  size  of  the  operation  and  available 
resources,  decide  on  the  number  of  observations  to  be  completed,  shifts  of  observations,  and  the  number  of  observers  to  be 
trained,  etc.  Within  each  location,  try  to  sample  as  many  different  technicians  or  ramp  employees  as  possible  to  capture  a 
representative  sample.  As  a  general  guideline  for  a  full  LOSA  implementation,  match  the  number  of  observations  per 
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location  to  the  relative  number  of  technicians  or  ramp  employees.  For  example,  30%  of  technicians  are  assigned  to 
Location  A,  then  approximately  30%  of  the  LOS  A  observations  should  occur  at  Location  A. 

(5)  Decide  on  the  Timing  of  Implementation.  The  implementation  team  needs  to  consider  several  factors  when 
scheduling  a  LOSA  implementation.  Given  all  the  personnel  involved,  a  LOSA  should  be  scheduled  to  fit  with  other 
operational  priorities.  For  example,  is  there  a  particular  time  in  the  year  when  more  observers  will  be  available?  Is  there  a 
better  time  for  the  scheduling  department  to  roster  observers  and  technicians  or  ramp  employees?  Also,  is  there  a 
particular  time  that  is  more  interesting  from  a  safety  or  operational  perspective?  Examples  of  interesting  times  include 
bad- weather  season,  peak-traffic  season,  and  after  the  introduction  of  an  operational  change  such  as  new  aircraft  or  a 
merger. 

Please  note  that  a  LOSA  should  not  be  implemented  immediately  after  a  major  incident  or  accident.  Your  organization 
will  be  in  a  heightened  state  of  awareness  at  this  time,  and  technicians  or  ramp  employees  will  be  overly  sensitive  to 
observation;  hence,  the  chances  of  getting  normal  data  will  be  diminished. 

Step  3:  Market  LOSA 


3.  Market  M- 
LOSA  &  R-LOSA 

(Multi-layer  & 
Multi-strategy) 


3.1  Mid 
Management 
Buy-in 

(e.g.,  brochure) 


3.2  Frontline 
Employees 
Awareness 
(e.g.,  poster, 
website) 


3.3  Union  or 
Employee  Group 
Leadership  Buy-in 
(e.g.,  face-to-face 
meeting) 


3.4  Business 
Partners' 
Awareness  (e.g., 
face-to-face 
meeting) 


Figure  6.  Step  3:  Market  M-LOSA  &  R-LOSA 


Organizations  are  naturally  resistant  to  change.  Therefore,  LOSA  will  need  to  be  sold  through  a  multi-level  and  multi¬ 
strategy  marketing  plan.  The  following  marketing  strategy  and  methods  are  recommended  for  the  various  audiences  (see 
Table  1). 


Table  1.  Multi-Level  and  Multi-Strategy  Marketing  Effort  for  LOSA  Programs 


Audience 

Strategy  and  Methods 

Middle  management 

Face-to-face  meeting,  brochure,  project  website 

Labor  union  and  employee  group 

Face-to-face  meeting,  project  website 

Frontline  employees 

Standardized  posters,  project  website,  letter/reminder 
from  the  implementation  team 

Business  partners 

Face-to-face  meeting,  project  website 

A  good  marketing  plan  should  clearly  define  the  safety  value  and  benefits  of  a  LOSA  program.  Examples  of  beneficial 
outcomes  from  other  M-LOSA  and  R-LOSA  programs  are  powerful  in  delivering  this  message.  For  upper  management, 
using  examples  to  quantitatively  illustrate  how  the  LOSA  program  can  reduce  cost.  The  section  “Step  12b:  Measure 
success  by  calculating  Return  on  Investment  (ROI)”  provides  a  system  for  such  calculations. 
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Second,  the  marketing  plan  should  clarify  how  the  LOSA  program  will  complement  existing  safety  programs.  Frontline 
employees  need  to  understand  how  they  will  fit  into  the  overall  picture.  LOSA  is  not  an  audit  process,  which  many 
organizations  already  do.  It  is  important  for  the  LOSA  implementation  team  (especially  external  members)  to  understand 
the  current  in-house  safety  programs,  including  all  voluntary  non-punitive  reporting  programs.  Then  the  implementation 
team  will  know  how  to  distinguish  and  contrast  LOSA  from  other  existing  safety  programs  when  promoting  LOSA. 
Management  will  be  concerned  whether  LOSA  and  existing  safety  programs  overlap  in  function,  methods,  and  outcomes. 
The  marketing  plan  should  explain  how  duplication  will  be  avoided  and  the  value  and  impact  that  the  LOSA  program  will 
bring  to  the  frontline  workforce.  Leadership  of  labor  unions  and/or  employee  groups  needs  to  communicate  information  to 
the  frontline  employees  and  promote  buy-in. 

Third,  the  marketing  plan  should  clarify  how  the  LOSA  program  will  fit  in  the  SMS  grand  scheme.  For  instance,  how 
data  will  feed  into  the  overall  safety  system,  how  data  are  going  to  be  integrated,  and  how  results  will  be  presented  to 
employees.  Conversations  regarding  LOSA  outcomes  should  be  customized  for  the  different  audiences. 

Communicate  often  and  broadly  during  the  implementation  and  align  the  communication  with  operational  areas. 

Paying  special  attention  to  raising  awareness  of  LOSA  and  obtaining  buy-in  at  the  beginning  of  a  LOSA  implementation 
will  assume  a  positive  outcome  (see  Figure  7).  It  is  important  to  invite  affected  business  partners  in  the  LOSA  initiative 
from  the  very  beginning.  For  example,  from  a  carrier’s  standpoint,  there  are  benefits  in  having  contractors  implement  their 
own  LOSA  process. 


Internalization 


Figure  7.  Importance  of  Raising  Awareness  and  Obtaining  Buy-In 

Source:  Workshop  on  Project  Management  for  Supply  Chain  Managers.  Center  for  Supply  Chain  Management  Studies. 

Saint  Louis  University.  August  5  &  12,  201 1 

Organizational  Change  Management  Form  (Appendix  A)  and  Resource  Requirements  RACI 
(Responsible/Accountable/Consulted/Informed)  Chart  (Appendix  B)  are  useful  tools  for  engaging  various  stakeholders 
during  planning  and  practicing  good  project  management  during  an  M-LOSA  or  R-LOSA  implementation.  The 
Organizational  Change  Management  Form  specifies  stakeholders’  involvement,  their  needs  and  demands  for 
communication  and  training,  and  resource  allocation.  The  Resource  Requirements  RACI  Chart  defines  how 
implementation  team  members  or  function  roles  share  the  responsibilities  for  each  deliverable. 
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Step  4:  Integrate  With  Existing  Safety  Programs/SMS 

The  implementation  team  should  conduct  a  gap  analysis  to  understand  the  existing  safety  programs,  SMS,  and  safety 
culture  (see  Figure  8). 
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Documentation 
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Figure  8.  Step  4:  Integrate  With  Existing  Safety  Program/SMS 

This  analysis  assesses  how  LOS  A  would  fit  into  the  existing  safety  programs.  It  will  also  help  determine  the  steps  to  be 
taken  in  moving  from  a  current  state  (i.e.,  prior-LOSA  integration)  to  a  desired  future  state  (i.e.,  post-LOSA  integration). 

A  gap  is  sometimes  spoken  of  as  "the  space  between  where  we  are  and  where  we  want  to  be."  Gap  analysis  is  an  essential 
activity  that  allows  the  implementation  team  to  determine,  document,  and  address  the  variance  between  LOSA 
requirements  and  current  safety  practices.  First,  review  and  describe  the  existing  safety  programs  by  listing  each 
program’s  attributes,  processes,  competencies,  and  performance  levels.  Second,  cross-list  factors  required  to  achieve  the 
future  objectives  (i.e.,  the  10  essential  LOSA  characteristics).  A  comparison  of  the  current  elements  of  the  safety  system 
and  the  new  LOSA  requirements  or  objectives  will  give  an  idea  of  whether  a  gap  exists.  If  there  is  a  gap,  there  will  be 
discrepancies  between  what  the  organization  wants  and  what  is  already  in  place.  This  comparison  may  take  the  form  of  a 
“Gap  -  Yes/No”  column  (see  Table  2)  to  identify  where  the  gaps  exist  for  each  element.  The  last  step  in  the  gap  analysis  is 
to  make  recommendations  by  identifying  the  items  or  solutions  needed  to  fill  the  gap,  if  a  gap  exists.  The  risks  and  effects 
of  introducing  or  implementing  the  item  or  solution  can  also  be  assessed.  Gap  analyses  often  results  in  action  plans,  which 
contain  task  assignments,  schedules,  resource  allocations,  and  evaluation  criteria  for  each  recommendation.  These  action 
plans  should  be  used  to  guide  the  LOSA  implementation. 

Here  is  a  list  of  methods  for  the  implementation  team  to  gather  information  for  the  gap  analysis: 

•  Focus  group  discussion  with  the  safety  department  (e.g.,  observation/audit  methodologies,  data  types), 

•  Review  operational  policy  and  procedures,  especially  disciplinary  procedures, 

•  Assess  SMS  maturity, 

•  Survey  the  safety  culture,  and 

•  Develop  LOSA  characteristic  chart. 

Table  2  is  an  example  of  a  template  that  can  be  used  to  conduct  a  gap  analysis. 
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Table  2.  A  Sample  Gap  Analysis  Table 


Gap  Analysis  Table 

Currently  in  place 

New  Program  Requirements 

Gap  Y/N 

Items  needed 

Process  and  performance  audits 
by  the  Safety  and  QA 
departments 

Peer-to-peer  assessment  through 
observations  of  normal  operations 

Y 

•  Observation  instrument 

•  Database  (software) 

•  Observer  training 

Non-punitive  voluntary  self- 
reporting  programs,  e.g.,  ASAP 

Peer-to-peer  assessment  of  normal 
operations  using  systematic 
observation  instrument 

Y 

•  Observation  instrument 

•  Database  (software) 

•  Observer  training 

Etc. 

If  you  are  interested  in  learning  more  about  gap  analysis,  please  refer  to  the  For  Additional  Reading  section  below. 


TIPS: 

A-  Delimit  the  initial  implementation  to  a  small,  manageable  sector. 

A-  Timeline  for  the  implementation  team: 

A  3  months  prior  to  a  planned  implementation:  review  readiness  checklist 

A  2  months  prior:  conduct  a  meeting  or  a  telecon,  go  through  the  checklist  of  to-dos,  figure  out  logistics 
A  1  month  prior:  conduct  in-person  meeting 

A  Promote  LOS  A  at  multi-levels  of  an  organization  through  various  strategies  and  methods. 

A  Emphasize  that  LOSA  is  not  for  disciplinary  purposes  and  that  the  observation  forms  do  not  include  any  personal 
information. 

A  Be  prepared  to  discuss  the  financial  rewards  of  a  LOSA  program,  especially  when  briefing  senior  management. 

A  Gap  analysis  is  an  essential  activity  to  determine,  document,  and  address  the  variance  between  LOSA 
requirements  and  current  safety  practice. 

KEY  REMINDERS: 

A  Assess  readiness  and  ensure  buy-in. 

A  Fundamentals  are  critical. 

A  Communication  to  everyone  through  awareness  training  or  other  channels  is  critical. 

A  Do  not  rush  a  LOSA  if  your  organization  is  not  ready.  Allow  time  for  your  safety  culture  to  mature  if  necessary. 

FOR  ADDITIONAL  READING: 

A-  LOSA  marketing  materials  and  templates  (e.g.,  awareness  posters)  are  available  at  the  “Marketing  Materials” 
section  of  the  project  website:  www.MRLOSA.com 

A-  FAA.  (2006).  Advisory  Circular:  Line  Operations  Safety  Audits  (AC  120-90):  Federal  Aviation  Administration. 

A-  IATA  Airport  Handling  Manual  (AHM)  -  3 1st  edition  www.iata.org/ps/publications/Pages/ahm.aspx  (see  IATA’s 
AHM  640). 

A-  ISO  9001  (2008)  Gap  Analysis  Tool,  www.praxiom.com/iso-gap.htm 

A-  Transport  Canada  (2010).  Gap  Analysis  Form:  www.tc.gc.ca/eng/civilaviation/publications/tpl4343-appendixb- 
610.htm 
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III.  LOSA  Infrastructure  Development  Phase 
Step  5a.  Construct  LOSA  Database 

5a.  Adapt/ 

Customize  LOSA 
Database 


LOSA 

1  Database 


Figure  9.  Step  5a:  Adapt/Customize  LOSA  Database 

Structured  Query  Language  (SQL),  designed  for  managing  data,  was  used  to  develop  the  M-LOSA  database  and  the  R- 
LOSA  database.  SQL  is  the  most  widely  used  database  language  and  offers  the  capability  to  insert,  query,  update  and 
delete,  create  database  structure,  and  provide  data  access  control.  With  modifications,  SQL  code  can  be  ported  between 
database  systems  (e.g.,  Microsoft  SQL  Server,  Oracle,  Teradata,  MySQL).  Each  organization  should  be  aware  of  this 
when  adopting  the  above  LOSA  databases,  and/or  when  integrating  the  LOSA  databases  with  other  safety  databases  inside 
its  organization.  The  LOSA  implementation  team  should  work  closely  with  their  database  administrators  to  adapt  and 
deploy  the  LOSA  database. 

The  LOSA  observations  can  be  kept  in-house  if  data  management  and  analysis  expertise  is  available  and  if  data 
security  can  be  assured.  Alternately,  the  data  can  be  sent  to  a  trusted  third  party  who  will  assume  responsibility  for  data 
input,  cleaning,  and  analysis.  The  decision  will  depend  on  an  organization’s  resources. 

The  LOSA  program  manager  should  organize  a  secure  site  for  the  data  and  oversee  the  receipt  of  the  observation  forms 
and  subsequent  data  analysis.  The  program  manager  should  be  able  to  protect  the  identity  of  the  observed  to  ensure 
complete  confidentiality  and  non-jeopardy  conditions.  Under  no  circumstances  should  it  be  possible  to  connect  individuals 
with  particular  observations. 


Staffing  needs 

A  database  administrator  and  a  data  analyst  are  essential  to  effectively  using  the  LOSA  databases.  To  adopt  the  above 
SQL  databases,  the  administrators  need  to  be  proficient  with  SQL  databases.  The  analysts  may  also  be  required  to  have 
relevant  professional  certification.  For  instance,  Microsoft  Certified  Database  Administrator  (MCDBA)  and  Microsoft 
Certified  Systems  Engineer  (MCSE)  certification  cover  Microsoft  SQL  Server  as  one  topic  area  in  the  certification 
examinations.  The  Microsoft  Certified  Technology  Specialist  (MCTS)  credential  enables  professionals  to  target  specific 
technologies,  and  is  generally  the  first  step  toward  the  professional-level  certifications.  SQL  Server  Specialization  is  one 
of  the  MCTS  specializations.  Another  key  responsibility  of  the  data  administrator  is  to  maintain  the  databases  by 
periodically  backing  up  the  SQL  Server  to  minimize  data  loss  in  case  the  data  become  corrupted.  Some  third-party  tools 
allow  backup  and  provide  additional  capabilities. 

The  data  analysts  should  be  familiar  with  an  organization’s  operations,  as  well  as  have  data  analysis  skills.  The  data 
analyst  and  report  writer  work  together  to  prepare  reports  of  the  findings  to  be  presented  to  management,  labor  groups,  and 
frontline  employees. 

An  organization  might  choose  a  third-party  analyst  if  expertise  is  not  available  in-house,  or  if  frontline  employees  have 
expressed  reservations  about  the  integrity  of  the  LOSA  implementation  or  the  objectivity  of  the  final  report. 
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Data  standardization 

Several  supplementary  techniques  ensure  that  good,  quality  standardized  data  are  used  in  a  LOSA  program. 

(1)  Observers  are  not  asked  to  evaluate  performance,  but  simply  to  observe  it.  From  a  data  standpoint,  this  is  the 
distinction  between  subjective  judgment  and  objective  observation.  For  example,  observers  are  asked  to  note  threats 
without  any  subjective  judgment  —  if  there  is  a  thunderstorm,  record  it;  if  there  is  a  malfunction  or  equipment  problem, 
record  it.  It  is  the  same  for  errors  and  undesired  operational  states.  Observers  also  note  the  frontline  employee’s  response 
to  threats,  errors,  and  undesired  states  and  the  outcome.  The  observer  is  not  assessing  the  employee’s  performance  or 
providing  a  subjective  evaluation.  The  observer  is  providing  an  overview  of  the  situation. 

(2)  The  data  verification  process  involves  checking  observation  forms  submitted  by  observers  against  the  written 
description  and  comments.  Observers  are  experts  at  describing  a  maintenance  or  ramp  procedure.  They  are  not  necessarily 
expert  at  assigning  codes  to  the  various  threats  and  errors,  especially  if  it  is  their  first  time  as  a  LOSA  observer.  Although 
training  is  given  on  how  to  code,  it  is  realistic  to  assume  that  observers  will  not  necessarily  retain  this  information 
perfectly.  The  comments  are  the  “fail-safe”  in  the  system  in  that  they  allow  the  analyst  to  read  the  events  of  the 
observation  and  match  them  to  the  observer’s  codes.  Because  the  analyst  should  be  an  expert  with  the  codes,  he/she  can 
add  any  codes  that  were  missed  and  recode  anything  that  might  not  be  correct.  Hence,  good  quality  comments  are  the 
ultimate  key  to  standardized  data.  Observers  provide  comprehensive  descriptions  and  comments,  and  the  analyst  ensures 
consistent  and  accurate  coding. 

(3)  Standardizing  the  LOSA  data  prior  to  analysis  also  involves  verifying  the  data  with  a  team  of  local  experts.  For 
example,  airline  personnel  familiar  with  the  maintenance  operation:  maintenance  managers,  representatives  of  the  airline’s 
maintenance  labor  group,  or  members  of  the  LOSA  implementation  team.  LOSA  observers  should  be  included  whenever 
it  is  possible.  The  group’s  task  is  to  review  and  verify  the  observations  against  current  manuals,  policies,  and  procedures. 
For  example,  an  observer  might  log  a  procedural  error  for  failure  to  lock  out  and  tag  out,  when  in  fact  there  is  no  such 
written  procedure  in  the  maintenance  manual.  The  “error”  would  then  be  deleted  from  the  database  or  marked  as  an 
omission  from  the  maintenance  manual.  The  data  verification  group  acts  as  a  check  on  the  threat  and  error  coding, 
ensuring  events  are  correctly  recorded  in  line  with  procedures  and  policies.  It  also  builds  ownership  in  the  results  and 
dispels  any  later  criticism  that  the  coding  was  not  accurate. 

Your  organization  may  consider  data-sharing  with  other  operators  to  enhance  the  power  of  data-derived  knowledge. 
Standardized  data  are  critical  for  possible  industry  sharing  and  comparison  using  one  singe  repository  over  the  long  run. 


Step  5b:  Conduct  Train-the-Trainer  Training 

Make  sure  that  staffs  accept  the  LOSA  concept  before  the  implementation  team  comes  in  to  deliver  LOSA  training. 
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Figure  10.  Step  5b:  Train-the-Trainer  Training 


Training  preparation  and  customization 

The  gap  analysis  provides  implementation  team  with  a  good  understanding  of  how  LOSA  is  different  from  existing 
safety  programs  and  the  contextual  information  for  adopting  LOSA  training.  LOSA  training  should  be  customized  for 
different  purposes  and  audiences.  In  particular,  the  implementation  team  should  supply  examples  and  cases  to  best 
customize  and  localize  the  LOSA  training.  The  following  set  of  PowerPoint  training  (see  Table  3)  and  corresponding  quiz 
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questions  are  available  for  download  from  www.MRLOSA.com.  This  training  is  recommended  to  be  adapted  to  suit  your 
organization’s  specific  needs  and  safety  environment. 


Table  3.  Recommended  LOSA  Training  Segments 


Specific  LOSA  Training 
Components 

Key  Contents 

Duration  and  Format 

1 .  LOSA  Awareness  T raining 

•  M-LOSA  &  R-LOSA  background 

•  TEM 

•  Brief  ROI 

Less  than  1  hour;  in- 
person  or  Computer- 
based  Training  (CBT) 

2.  LOSA  Train-the-Trainer 

•  LOSA  Awareness  training  (full) 

•  TEM 

•  LOSA  products 

•  Implementation  process 

•  Forms/Scenarios  (Maintenance 
LOSA  and/or  Ramp  LOSA) 

•  Database 

•  Reporting 

16-18  hours;  in  person 
with  hands-on 
observation  and 
database  practice  and 
“teach-back”  session 

3.  LOSA  Observer  Training 

•  LOSA  Awareness  training  (full) 

•  TEM 

•  LOSA  products 

•  Implementation  process 

•  Forms/Scenarios  (Maintenance 
and/or  Ramp  LOSA) 

•  Database 

•  Observation  protocol 

16  hours;  in  person  with 
hands-on  observation 
and  database  practice 

4.  LOSA  Database  Training 

•  Set  up  LOSA  database 

•  Maintain  LOSA  database 

•  Customization 

2  hours;  in  person  with 
hands-on  database 
practice 

5.  LOSA  Reporting  Training 

•  Reporting 

1-2  hours;  in  person  with 
hands-on  database 
practice 

Each  training  segment  is  designed  for  differing  audiences.  Trainers  should  carefully  consider  the  needs  of  each  group 
when  selecting  and  tailoring  the  training.  Certain  training  segments  are  suggested  for  the  various  audiences  (see  Table  4). 
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Table  4.  Target  Audiences  of  Various  LOSA  Training  Segments 


Targeted  Audience 

Specific 

LOSA 

Training 

Frontline 

Employees 

(non¬ 

observers) 

LOSA 

Observers 

LOSA 

Trainers 

LOSA 

Program 

Managers 

LOSA 

Analysts 

Management  I 

>  usiness 

Partners** 

LOSA 

Awareness 

Training 

X 

X 

X 

X 

X 

X 

X 

LOSA  for 
the  C-Suite 
and 

Management 

X 

LOSA 

Train-the- 

trainer 

X 

X 

LOSA 

Observer 

Training 

X 

X 

LOSA 

Database 

Training 

X* 

X 

X 

X 

LOSA 

Reporting 

Training 

X 

X 

X 

*At  some  organizations,  LOSA  observers  may  enter  data  themselves. 

**Depending  on  how  involved  the  business  partners  are  in  the  LOSA  initiative,  they  may  participate  in  all  or  some  of 
the  available  LOSA  training. 


LOSA  awareness  training 

LOSA  awareness  training  is  to  ensure  employees  know  the  fundamentals  of  LOSA,  its  purpose,  and  how  it  is  different 
from  other  safety  programs,  especially  other  voluntary,  non-punitive  reporting  programs.  All  frontline  employees  should 
understand  from  participating  in  this  training  that  LOSA  is  a  voluntary  program  —  they  have  the  right  to  refuse  being 
observed. 

LOSA  awareness  training  will  demonstrate  values  to  the  employees  through  a  circular  process:  (1)  employees  provide 
LOSA  data  by  volunteering  to  become  observers  or  to  be  observed;  (2)  employees  informed  how  data  are  used  and  what 
changes  have  occurred;  and  (3)  employees  understand  how  LOSA  benefits  them  at  individual  and  organizational  levels 
and  become  more  motivated  to  participate.  Thus,  LOSA  may  create  a  domino  effect  (from  observing  the  changes  to 
making  the  changes)  by  positively  leveraging  peer  pressure. 

Training  frontline  employees  may  generate  changes  in  safety  culture.  Safety  cultures  consist  of  shared  beliefs, 
practices,  and  attitudes  that  exist  in  an  organization.  It  is  highly  influenced  by  a  number  of  factors  such  as  employee 
training  and  motivation  and  employee  involvement  or  “buy-in.”  Awareness  training,  along  with  a  “kick-off’  celebration, 
can  be  used  to  involve  the  entire  site  and  announce  it  is  a  “new  day”  and  seek  buy-in  for  any  new  procedures  and 
programs. 

LOSA  awareness  training  needs  to  address  the  following  topics: 

•  Provide  an  overview  of  SMS  and  its  safety  programs  (e.g.,  ASAP), 

•  Provide  a  general  background  of  the  M-LOSA  &  R-LOSA  project  and  introduce  the  project  website. 

•  Explain  how  LOSA  fits  with  the  other  components  of  SMS. 

Upon  completion  of  the  LOSA  awareness  training,  participants  should  be  able  to 
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•  Define  LOS  A, 

•  Understand  the  purpose  of  LOS  A, 

•  Identify  the  goals  of  LOS  A, 

•  Describe  the  characteristics  of  LOSA,  and 

•  Outline  the  steps  necessary  to  integrate  LOSA. 

TEM  is  the  underlying  framework  of  LOSA  observations.  Training  on  the  TEM  model  can  be  likened  to  “defensive 
driving”  for  the  frontline  employees.  Just  as  when  on  the  road  drivers  exercise  TEM-related  techniques  (e.g.,  being  alert  to 
other  drivers  on  the  road),  frontline  employees  minimize  safety  risks  by  employing  a  proactive  safety  philosophy.  Error 
management  is  considered  to  be  a  critical  component  of  learning  and  adapting.  Hence,  training  can  increase  awareness  of 
TEM  and  result  in  better  managed  threats  and  fewer  errors. 

The  TEM  training  should  enable  the  participants  to: 

•  Describe  the  TEM  process, 

•  Describe  what  threats  are  and  how  to  identify  them, 

•  Describe  what  errors  are  and  how  to  identify  them,  and 

•  Explain  the  different  types  of  error  outcomes. 

It  is  highly  recommended  to  use  aviation  examples  to  illustrate  components  and  inter-relationships  of  the  various 
components  in  the  TEM  model.  In  the  sample  training,  Air  Transat  Flight  236,  was  used  as  a  concrete  example  to  explain 
the  TEM  model.  Each  organization  should  further  customize  TEM  training  to  include  events  that  are  familiar  to  the 
training  participants  and  specific  to  a  particular  operating  environment  where  the  target  participants  work. 

Including  the  TEM  component,  the  LOSA  awareness  training  runs  for  less  than  one  hour.  The  LOSA  awareness 
training  (including  SMS  and  TEM)  may  be  identical  for  both  M-LOSA  and  R-LOSA  programs.  Participants  to  the  train- 
the-trainer  and  observer  training  can  join  together  for  the  foundational  information,  and  then  break  away  for  the  specific 
technical  aspects  of  M-LOSA  and  R-LOSA  observations. 


Train-the-trainer  training 

The  train-the-trainer  training  takes  16  to  18  hours  over  two  to  three  consecutive  days  to  complete.  It  should  be 
conducted  in  person  with  hands-on  observation  and  database  practice,  as  well  as  a  “teach-back”  session. 


Trainer  selection  criteria 

It  is  generally  agreed  that  face-to-face  training  is  the  most  desirable,  and  CBT  would  be  a  minimum  requirement  for 
refresher  training.  LOSA  train-the-trainer  training  should  be  a  self-funded  activity. 

Minimum  qualifications  for  trainer  candidates  are  preferably: 

•  A  technical  training  background  in  the  area  of  the  planned  LOSA  activity  (e.g.,  maintenance  or  ramp),  and 

•  Human  factors  experience. 

Trainer  candidates  should  be  representatives  from  the  aviation  community.  They  should  be  subject  matter  experts  with 
facilitation  skills  who  can  relate  to  the  training  attendees.  Possible  candidates  for  the  trainers  are  experienced  trainers  at 
your  organizations. 

Refresher  training  for  the  LOSA  trainers  may  be  provided  on  an  as-needed  basis.  Each  organization  will  determine  a 
proper  frequency  and  monitoring  system  for  refresher  training  as  the  M-LOSA  and  R-LOSA  programs  progress  industry 
wide.  One  important  goal  of  the  trainer  refresher  training  is  to  re-calibrate  how  they  deliver  LOSA  training  and  conduct 
LOSA  observations  to  maintain  quality  and  consistency  throughout  the  industry. 


How  to  train  the  trainer 

Once  a  LOSA  trainer  candidate  is  identified,  the  implementation  team  should  send  him/her  a  welcome  letter  detailing 
what  to  expect  as  a  LOSA  trainer  about  a  month  prior  to  the  LOSA  train-the-trainer  training. 

The  train-the-trainer  model  reflects  the  theory  that  people  who  train  others  generally  recall  90%  of  what  they  teach  and 
that  people  often  learn  new  information  through  trusted  social  networks.  The  model  has  been  applied  to  different  types  of 
programs,  in  which  a  core  group  of  people  are  provided  with  the  skills  and  training  to  teach  others  about  a  specific 
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program  or  topic.  The  LOSA  train-the-trainer  training  is  essentially  an  expanded  version  of  observer  training  with  a 
“teach-back”  session.  How  to  conduct  observer  training  will  be  explained  in  great  detail  in  “Step  6:  Observer  Training”  of 
this  guideline.  It  is  important  to  have  a  “teach  back”  session  to  make  sure  that  the  trainers  understand  and  are  capable  of 
relaying  their  knowledge.  The  teach-back  session  is  a  way  to  confirm  that  the  trainers  understand  the  materials  and  can 
teach  them  back  to  an  audience.  This  practice  is  expected  to  further  enhance  trainers’  mastery  of  curriculum  material. 
“Teach-back”  is  also  known  as  the  “show  me”  method  or  “closing  the  loop,”  which  is  one  of  the  easiest  tools  with  which 
to  close  the  gap  in  training. 


LOSA  database  training 

The  LOSA  database  training  takes  approximately  two  hours.  It  should  be  conducted  in  person  with  hands-on  database 
practice.  The  trainer(s)  should  provide  a  stand-alone  training  database  and  a  practice  dataset,  which  could  be  loaded  on 
(an)  individual  computer(s)  for  training  purposes.  This  would  allow  a  trainee  to  experience  the  inputs  and  outputs  without 
affecting  the  data  in  a  production  database. 

The  LOSA  database  training  should  cover  the  following  recommended  topics: 

•  Logging  in,3 

•  Getting  help, 

•  Setting  up  accounts, 

•  LOSA  observations,  and 

•  Editing  data  from  previous  observations. 


LOSA  report  training 

The  LOSA  report  training  takes  approximately  one  to  two  hours.  It  should  be  conducted  in  person  with  hands-on 
database  practice.  The  trainer(s)  should  provide  a  stand-alone  training  database  and  a  practice  dataset,  which  could  be 
loaded  on  (an)  individual  computer(s)  for  training  purposes.  This  would  allow  a  trainee  to  experience  generating  various 
reports. 

The  LOSA  report  training  should  cover  the  following  recommended  topics: 

•  Reporting  capabilities, 

•  How  to  run  generic  reports,  and 

•  How  to  run  custom  reports. 


Refresher  training 

Observers  s  hould  undergo  r  ecurrent  t  raining  e  very  12  t  o  2  4  m  onths  t  o  en  sure  th  ey  are  calib  rated  with  th  e  LOSA 
standards.  Table  5  outlines  the  contents,  frequency,  duration,  and  format  of  refresher  training. 


3  Assuming  a  database  administrator  has  already  downloaded  and  installed  the  LOSA  database. 
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Table  5.  Recommended  Refresher  Training  for  Different  Segments  of  LOSA  Training 


Specific  LOSA  Training 

Contents  of  the  Refresher  Training 

Frequency,  Duration,  and 
Format 

LOSA  Awareness  Training 

•  M-LOSA  &  R-LOSA  background 

•  TEM 

•  Brief  ROI 

Every  other  year;  less  than  1 
hour;  CBT 

LOSA  Train-the-Trainer 

•  Implementation  process 

•  Forms/Scenarios  (Maintenance  LOSA 
and/or  Ramp  LOSA)* 

•  Database 

•  Reporting 

•  Q&A 

Every  other  year;  4-6  hours; 
in  person  with  hands-on 
observation  and  database 
practice  and  “teach-back” 
session 

LOSA  Observer  Training 

•  Implementation  process 

•  Forms/Scenarios  (Maintenance  and/or 
Ramp  FOSA) 

•  Database 

•  Observation  protocol 

•  Q&A 

Every  year  or  every 
implementation  run  (whichever 
occurs  sooner);  4-6  hours;  in 
person  with  hands-on 
observation  and  database 
practice 

LOSA  Database  Training 

•  Maintain  FOSA  database 

•  Customization 

•  Q&A 

Every  other  year;  2  hours;  in 
person  with  hands-on  database 
practice 

LOSA  Reporting  Training 

•  Reporting 

•  Q&A 

Every  other  year;  1-2  hours; 
in  person  with  hands-on 
database  practice 

*M-LOSA  and  R-LOSA  programs  have  separate  training  packages.  M-LOSA  observer  training  has  two  variations: 
base  maintenance  and  line  maintenance.  These  training  segments  provide  scenario-specific  training  for  each  work  area. 
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Step  5c:  LOS  A  Website 


LOSA  Promo 
Channels  (e.g., 
website, 
bulletin  board, 
newsletter) 


5c.  Establish  and 
Maintain  a  LOSA 
Website 


Figure  11.  Recommended  Refresher  Training  for  Different  Segments  of  LOSA  Training 


It  is  beneficial  to  create  a  virtual  LOSA  site  or  dedicate  a  section  of  an  existing  organization-wide  virtual  site  to  LOSA 
programs.  This  site  can  be  used  to  promote  awareness  of  the  LOSA  program,  prepare  the  organization  for  LOSA 
observations,  and  present  LOSA  outcomes  and  action  items  to  the  workforce.  A  virtual  site  can  also  be  used  by  the  LOSA 
trainers,  observers,  analysts,  the  program  manager,  and  others  to  store  and  distribute  observation  forms  and  training 
materials.  Observers  can  submit  observations  via  the  site  to  a  secured  database.  Analysts  and  the  program  manager  can 
analyze  and  run  reports  on  the  site. 


TIPS: 

A-  Instructors  need  to  understand  the  difference  between  LOSA  and  other  similar  and  dissimilar  safety  programs  at  a 
given  organization. 

A-  Each  segment  of  LOSA  training  should  start  with  clearly  stated  learning  objectives.  Desired  learning  outcomes 
should  be  reviewed  at  the  end  to  summarize  the  training. 

A-  Provide  handouts  for  trainees  to  follow  along  and  take  notes,  as  well  as  review  and  reference  after  the  training. 

A-  CBT  should  be  used  to  complement,  not  replace,  in-person  training. 

A-  Document  and  share  lessons  learned  from  other  LOSA  programs  and  implementations,  so  the  new 
implementation  can  be  informed  and  prepared. 

KEY  REMINDERS: 

A-  Ensure  buy-in. 

Fundamentals  are  critical. 

A-  Communication  to  everyone  is  critical. 

FOR  ADDITIONAL  READING: 

A-  Maintenance  &  Ramp  LOSA  training  materials  and  templates  are  available  at  the  “Training”  section  of  the 
project  website:  www.MRLOSA.com 

A-  Bloom,  B.  (1956).  Bloom’s  Taxonomy  of  Learning  Domains  www.nwlink.com/-donclark/hrd/bloom.html 

A-  Types  of  Questions  based  on  Bloom’s  Taxonomy: 

www2.honolulu.hawaii.edu/facdev/guidebk/teachtip/questype.htm 

A-  Clarke,  D.  (2009)  Transfer  of  Learning:  A  brief  and  clear  introduction  -  including  an  explanation  of  why  transfer 
during  a  training  event  is  important. 

www.nwlink.com/-donclark/hrd/learning/transfer.html 

A-  FAA  Human  Factors:  Training  Transfer  Online  Reference: 
www.hf.  faa.  gov/webtraining/T  raining/T  raining023  .htm 

A-  Phillips,  J.  J.  (1991).  Handbook  of  Training  Evaluation  and  Measurement  Methods,  Second  Edition.  Houston, 

TX:  Gulf  Publishing  Company. 

A-  Sierra,  K.  &  Russell,  D.  (2005).  Ten  Tips  for  New  Trainers/Teachers. 

headrush.typepad.eom/creating_passionate_users/2005/07/ten_tips_for_ne.html 
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Step  6:  Observer  Training 

Observer  selection  criteria 

LOS  A  observer  candidates  should  possess  the  following  characteristics: 

1 .  Proficiency  in  the  field  they  are  observing  (i.e.,  maintenance  or  ground  operations). 

2.  Familiarity  with  the  operating  procedures.  LOS  A  may  help  potentially  identify  a  discrepancy  between  practice  and 
standard  operating  procedures. 

3.  Respected  frontline  employees,  but  not  a  QA  person.  If  a  QA  person  were  selected  to  be  a  LOS  A  observer,  he/she 
might  be  able  to  switch  roles  from  auditing  to  observing,  but  it  would  be  difficult  for  the  employees  to  switch  their 
perception  from  the  auditor  function. 

4.  No  permanent  LOS  A  observers.  They  will  be  rolled  back  into  their  normal  jobs  to  stay  current  with  changing  and 
evolving  procedures  and  practices.  Because  of  their  LOSA  observation  experience,  these  staffs  often  become  “safety 
advocates”  in  their  work  group.  New  observers  also  provide  a  fresh  perspective  and  are  likely  to  notice  threats  and 
errors  that  were  previously  overlooked. 

Some  organizations  employ  a  selection  procedure  whereby  management  and  the  labor  or  employee  group  each  put 
forth  a  list  of  acceptable  observers,  and  selections  are  made  from  the  personnel  that  appear  on  both  lists.  The  observer 
team  can  include  a  small  number  of  non-technicians  or  non-ramp  employees,  as  long  as  they  can  anticipate  and 
understand  operational  tasks  and  their  surrounding  context.  However,  the  majority  of  the  team  should  be  active  frontline 
employees. 

The  number  of  observers  needed  depends  on  the  size  of  the  assessment  and  the  observers’  workload.  More  data  are 
better.  Considering  all  the  constraints,  the  LOSA  implementation  team  should  commit  to  drawing  a  representative  sample 
of  its  fleet.  There  is  substantial  work  involved  in  completing  an  observation  form  and  providing  detail-rich  comments  for 
each  observation;  therefore,  an  organization  needs  to  commit  adequate  resources  to  conduct  LOSA  observations. 

Statistical  formulas  and  web-based,  sample-size  calculators  are  available  in  the  For  Additional  Reading  section.  You  can 
use  them  to  determine  how  many  people  you  need  to  observe  to  get  results  that  reflect  the  target  population  as  precisely  as 
needed.  You  can  also  find  the  level  of  precision  you  have  in  an  existing  sample. 

In  airlines  that  operate  more  than  one  base,  observers  should  sometimes  be  scheduled  to  observe  across  bases  other 
than  their  own,  if  the  observers  are  known  to  the  workforce  at  those  bases.  This  adds  value  to  the  process  in  that  the 
observer  looks  more  at  the  “big  picture”  rather  than  the  fine  detail.  For  similar  reasons,  experience  has  shown  that  using  a 
small  percentage  of  external  observers  drawn  from  technicians  with  LOSA  experience  at  other  airlines  adds  value  in  the 
form  of  a  “control  group”  for  observations.  External  observers  will  normally  attend  the  airline  LOSA  training  and  will 
need  to  familiarize  themselves  with  airline  procedures.  They  are  particularly  useful  in  picking  up  systemic  and 
organizational  threats  to  which  airline  staff  are  often  “blind”  due  to  familiarity. 

How  to  train  the  observers 

Required  contents  in  the  observer  training  should  include  the  following  segments.  The  first  segment  should  provide 
learning  objectives,  a  brief  history  of  LOSA,  TEM,  how  LOSA  can  be  integrated  into  current  safety  programs,  and  lessons 
learned.  The  second  segment  should  explain  how  to  conduct  observations  through  scenarios,  and  a  minimum  two-hour 
practice  run  in  the  hangar  or  shop,  at  the  line,  or  on  the  ramp.  Observers  should  practice  with  scripted  scenarios  or  videos 
until  they  are  confident  they  can  use  the  observation  form  correctly.  At  this  point,  they  can  be  dispatched  to  the  hangar, 
line,  shop,  or  ramp.  During  the  practice  observation,  if  there  are  any  questions  or  doubts,  observers-in-training  need  to 
immediately  ask  the  trainers  what  to  do  and  how  to  respond.  It  is  recommended  that  observers  be  brought  back  in  after  one 
or  two  observations  to  discuss  their  observations,  correct  any  misperceptions,  and  coach  them  on  areas  that  require 
clarification.  In  the  third  segment,  trainer(s)  should  lead  the  training  class  to  review  observers-in-training’ s  observation 
notes  and  coding,  discuss  and  explore  questions  and  inconsistencies  in  observation  and  coding,  as  well  as  assess  the  inter¬ 
observer  consistency.  The  fourth  segment  should  focus  on  data  entry,  reporting  using  the  database,  and  hands-on  practice 
using  a  provided  sample  database  or  data  collected  from  the  practice  run.  At  the  end  of  the  observer  training  (the  fifth 
segment),  the  trainers  should  guide  participants  to  review  the  learning  objectives  and  assess  if  each  of  the  objectives  has 
been  satisfactorily  met  and  conduct  a  learning  assessment.  A  learning  curve  should  be  expected.  Usually,  the  first  two 
segments  and  half  of  the  third  segment  take  place  on  training  day  one,  while  the  rest  take  place  during  the  second  day’s 
training. 


23 


Across  different  training  segments,  observer  training  should  emphasize  the  following  five  topics: 

1.  LOS  A  Rationale  and  Etiquette 

Observers  will  likely  have  a  rudimentary  understanding  of  LOSA  when  selected  to  conduct  observations,  but  they  will 
need  to  fully  understand  the  safety  rationale  for  conducting  a  LOSA  at  their  organization.  A  “big  picture”  perspective  will 
help  observers  understand  the  “why”  of  LOSA.  Also,  the  observers  will  be  ambassadors  for  LOSA  while  observing  hangar 
maintenance,  line  maintenance,  shop  maintenance,  or  ramp  operations.  It  is  important  that  they  can  explain  the  process 
fully,  answer  any  questions  that  the  frontline  employees  may  have,  and  address  any  apprehensions. 

Specifically,  the  observers  need  to  understand  the  safety  rationale  for  normal  operations  monitoring  —  a  discussion  of 
predictive  versus  reactive  and  proactive  safety  strategies  is  recommended.  The  observers  also  need  to  know  how  the  data 
collected  from  the  LOSA  will  be  used  to  understand  strengths  and  weaknesses  in  the  operations.  An  overview  is 
recommended  of  the  whole  process  from  observations  to  data  cleaning  and  analysis,  to  the  diagnostic  report,  and  the 
development  of  targets  for  enhancement. 

Of  course,  the  observers  will  also  need  to  know  the  “how”  of  LOSA,  specifically  the  etiquette  associated  with  being  a 
LOSA  observer.  An  observer  needs  to  learn  how  to  approach  a  technician  or  a  ramp  employee,  how  to  ask  permission  to 
observe  a  procedure,  how  to  walk  away  so  that  the  technicians  or  ramp  employees  can  discuss  it,  and  to  accept  without 
question  any  frontline  employee’s  decision  to  deny  the  opportunity  to  observe.  The  observer  can  also  carry  a  copy  of  the 
letter  of  endorsement  jointly  signed  by  management  and  the  labor  or  employee  group  to  show  any  interested  frontline 
employee,  if  appropriate. 

During  a  LOSA,  the  observer’s  behavior  is  best  summarized  as  a  “fly  on  the  wall.”  The  observer  needs  to  be 
unobtrusive,  yet  responsive  to  any  queries  the  technicians  or  ramp  employees  may  have  about  the  LOSA  process.  LOSA 
observers  should  be  trained  to  accept  their  role  as  observers,  not  evaluators  —  they  are  NOT  auditors.  LOSA  observers 
will  observe  errors  and  undesired  states  as  part  of  their  observations;  however,  they  should  only  intervene  if  they  know 
that  the  aircraft  will  not  be  airworthy  or  if  aircraft  damage  or  employee  injury  is  about  to  occur.  A  helpful  rule  of  thumb  is 
to  ask  observers  to  think  of  themselves  as  a  guest  at  another  hangar,  line,  shop,  or  station.  This  helps  distance  the  observer 
from  the  frontline  employees,  while  still  being  able  to  politely  point  out  airworthiness  issues,  potential  aircraft  damage,  or 
potential  employee  injury  concerns. 

The  LOSA  observer  is  the  only  person  in  the  LOSA  process  that  will  know  the  identities  of  the  people  who  were 
observed,  so  it  is  essential  that  observers  are  reminded  throughout  the  training  of  their  responsibilities  in  this  regard. 
Anonymity  is  paramount,  and  the  observed  employees’  names  should  not  be  discussed  with  anyone,  not  even  other 
observers.  Experience  has  shown  that  at  the  end  of  an  observation,  technicians  or  ramp  employees  might  ask  the  observer 
to  “debrief’  their  performance.  In  these  circumstances,  it  is  essential  that  the  observer  politely  decline  the  invitation.  This 
emphasizes  the  concept  that  the  observer  is  not  there  to  evaluate  the  frontline  employees,  merely  to  record  observations. 

Experience  at  some  organizations  has  shown  the  benefits  of  a  post-observation  discussion  with  the  technicians  or  ramp 
employees.  The  sole  purpose  of  this  discussion  is  to  gather  additional  demographic  information  and  offer  a  two-way 
communication  opportunity  to  clarify  questions  that  either  observers  or  the  employees  being  observed  may  have.  This 
discussion  is  voluntary,  so  maintenance  technicians  or  ramp  employees  can  refuse  to  answer  any  questions  they  do  not  feel 
comfortable  addressing.  In  ramp  LOSA,  observers  often  have  to  stand  far  away  from  the  frontline  employees  because  of 
high  activity  levels,  so  the  observers  might  miss  some  contextual  details  and  cues.  Immediately  after  each  observation  is 
the  best  time  to  speak  with  the  frontline  employees.  A  discussion  after  each  observation  may  be  a  natural  and  polite  way  to 
end  the  observation.  Please  note  the  post-observation  discussion  is  not  an  investigation.  LOSA  observations  focus  on 
observed  threats  and  errors,  and  the  post-observation  discussion  helps  clarify  the  observable  threats  and  errors.  It  is  not 
intended  to  identify  unobservable  threats,  but  the  post-observation  discussion  may  reveal  some  relevant  information  that 
confirms  the  existence  of  unobservable  threats. 

The  observers  need  to  be  trained  and  calibrated  to  carry  out  the  post-observation  discussion  using  a  pre-scripted 
protocols.  Ask  the  questions  in  a  diplomatic  way  (e.g.,  when  observing  multiple  ramp  personnel  not  using  knee  pads  ask, 
“Why  do  people  not  use  knee  pads  at  this  station?”  instead  of  “Why  didn’t  you  put  on  knee  pads?”).  An  organization  can 
decide  if  it  wants  to  adopt  the  post-observation  discussion  based  on  its  safety  culture,  communication  style,  and  labor- 
management  dynamics.  There  is  certainly  an  advantage  to  gathering  additional  information.  However,  at  some 
organizations,  the  post-observation  discussion  may  be  perceived  as  investigative  consequently  altering  the  non-punitive 
tone  of  LOSA  and  potentially  killing  the  program. 

In  summary,  LOSA  observers  should  act  in  an  unobtrusive  and  consistent  manner  so  that  technicians  or  ramp 
employees  have  a  similarly  positive  experience  with  LOSA.  This,  in  turn,  will  favorably  affect  their  receptivity  to  the  final 
results  and  outcomes. 
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2.  TEM  Concepts 

Since  TEM  is  the  basis  of  LOSA,  observers  should  be  able  to  define,  distinguish,  and  identify  threats,  errors,  and 
undesired  states.  This  is  best  achieved  with  a  mixture  of  lecture,  case  studies,  and  review.  Lecture  material  should  include 
multiple  examples  of  each  type  of  threat,  error,  and  undesired  operational  state.  Case  studies  can  take  the  form  of  scripted 
vignettes  and/or  actual  accident  and  incident  report  excerpts.  The  distinction  between  threats,  errors,  and  undesired  states 
becomes  clear  with  practice.  Observers  are  usually  able  to  correctly  distinguish  examples  of  all  three  categories  in  two 
days  or  fewer  of  classroom  involvement. 

3.  Company  Policies  and  Procedures 

Observers  need  to  be  current  with  company  policies  and  procedures  so  as  to  observe  procedural  adherence  or 
deviations.  Selecting  active  frontline  employees  from  your  organization  is  one  way  to  ensure  this.  Spending  some  time  in 
the  classroom  reviewing  relevant  procedures  allows  all  observers  to  get  “up  to  speed”  on  what  they  will  be  observing. 
Observers  should  also  be  encouraged  to  review  the  manuals,  SOPs,  and  work  cards  as  homework. 

4.  Observation  Form 

Each  observer  should  be  able  to  correctly  complete  and  submit  a  LOSA  observation  form.  Hence,  observers  should  see 
the  observation  forms  and  threat  or  error  codes  as  soon  as  possible  after  the  training  begins  so  that  they  have  a  clear  sense 
of  what  is  expected  of  them.  With  the  observation  form  in  hand,  the  observers  can  be  led  through  the  various  sections,  and 
then  practice  using  scenarios. 

If  the  observation  form  is  software-based,  time  should  be  spent  ensuring  that  all  observers  have  the  necessary  computer 
skills  to  open  the  forms,  enter  and  edit  data,  and  submit  the  observation.  Mastering  these  skills  in  the  classroom  will  avoid 
potential  loss  of  data  later  in  the  LOSA  due  to  computer  errors.  Observers  should  have  the  name  of  an  IT  person  inside 
their  company  to  contact  in  the  event  of  computer  problems. 

5.  Written  Description 

Observers  need  training  in  writing  comments.  If  they  understand  the  concepts  underlying  the  observation  forms  as  well 
as  the  diagnostic  rationale  for  conducting  a  LOSA,  the  observers  will  realize  that  good-quality  comments  are  imperative. 
The  observation  forms  should  contain  several  prompts  to  help  the  observer  provide  sufficient  detail,  and  observers  should 
be  encouraged  to  “overwrite”  the  observation,  rather  than  provide  too  little  detail.  In  particular,  observers  need  to  record 
threats,  errors,  and  error  outcomes,  including  the  context  in  which  they  happened,  and  the  frontline  employee’s  responses. 
Observers  are  selected  because  they  are  experts  at  understanding  maintenance  or  ground  operations,  and  this  expertise  is 
best  expressed  in  detailed  comments.  As  long  as  the  observer  provides  detailed  comments  of  the  observation,  any  coding 
oversights  can  be  remedied  later  in  the  data-cleansing  process  (i.e.,  identifying  incomplete,  incorrect,  inaccurate,  irrelevant 
parts  of  the  data  and  then  cleaning  this  “dirty”  data). 


Observer  standardization 

Standardization  refers  to  the  need  to  be  sure  that  observational  details  are  recorded  in  a  systematic  and  consistent 
fashion.  In  LOSA,  standardization  is  a  multi-step  process  that  involves  calibrating  the  observers  and  conducting  follow-up 
data-cleansing  and  coding  of  completed  observations.  Both  inter-rater  reliability  and  intra-rater  reliability  need  to  be 
assessed. 

The  first  step  in  any  observer  standardization  is  quality  training.  To  be  sure  observers  understand  the  concepts,  group 
discussions  are  encouraged.  Focusing  on  the  finer  points  of  the  model  and  the  observation  forms  can  help  calibrate  the 
observers  to  a  common  standard.  A  test  can  be  administered  at  the  end  of  the  training  to  ensure  that  all  observers  have 
grasped  the  necessary  knowledge  and  can  demonstrate  the  required  competencies  as  specified  by  the  training  objectives.  If 
observers  complete  this  test  satisfactorily,  they  can  be  released  to  the  line  to  complete  one  or  two  trial  observations. 

The  LOSA  project  manager  or  trainer  should  schedule  time  with  observers  to  discuss  their  trial  observations.  If  the 
observer  is  competent,  as  evidenced  particularly  by  the  quality  of  the  coding  and  comments  (notes),  the  observations  can 
be  retained  and  used  in  the  LOSA,  and  the  observer  can  be  sent  back  out  to  complete  observations.  If  the  quality  of  the 
coding  and  comments  is  poor  (e.g.,  inaccurate  coding,  lacking  sufficient  detail  in  comments),  the  trainer  can  work  with  the 
observer  to  help  correct  misunderstandings  and  draw  out  missing  information.  If  the  observer  has  forgotten  details  and 
cannot  recreate  the  observation,  the  observation  should  be  discarded,  and  the  observer  sent  out  to  complete  another  trial 
observation.  It  is  the  LOSA  coordinator’s  decision  to  drop  any  observer  from  the  observation  team  if  that  person  fails  to 
meet  the  required  standard.  For  this  reason,  it  can  be  a  good  idea  to  initially  recruit  and  train  more  observers  than  needed 
to  allow  for  attrition,  illness,  and  scheduling  conflicts. 

Handouts  are  recommended  for  training  participants  to  take  notes  during  the  training  and  use  as  a  reference  in  the 
future.  The  handouts  should  include  a  written  protocol  on  how  to  conduct  LOSA  observations.  For  example,  how  to: 

•  approach  peers  at  work,  make  introduction,  explain  the  observation  process,  and  ensure  that  everyone 
understands  the  non-punitive  nature  of  the  assessment; 
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•  answer  the  typical  questions  from  the  frontline  employees; 

•  obtain  frontline  employees’  permission  for  conducting  observations; 

•  physically  position  to  best  observe  a  maintenance  or  ramp  activity;  being  a  “fly-on-the-wall;” 

•  use  scenarios  to  illustrate  when  to  intervene; 

•  carry  out  a  peer-to-peer  discussion  once  a  LOSA  observation  is  completed. 

Observers  have  to  use  the  forms  during  the  observation,  even  after  becoming  familiar  with  the  process  and  contents. 
The  observer  training  takes  approximately  16  hours  to  complete  over  two  to  three  consecutive  days.  It  should  be 
conducted  in  person  with  hands-on  observation  and  database  practice.  Upon  completion  of  the  LOSA  observer  training,  an 
observer  should  be  able  to: 

•  knowledgeably  and  confidently  explain  the  rationale  and  process  for  conducting  a  LOSA  at  the  organization; 

•  enact  the  LOSA  observer  etiquette  in  a  professional  and  consistent  manner; 

•  understand  the  theoretical  framework  of  the  observation  forms.  If  the  tool  is  based  on  TEM  concepts,  the 
observer  should  be  able  to  define,  distinguish,  and  identify  threats,  errors,  and  undesired  states; 

•  demonstrate  knowledge  of  company  policies  and  procedures; 

•  use  the  observation  forms  accurately  and  comprehensively; 

•  write  detailed  and  comprehensive  comments  from  which  others  will  be  able  to  understand  the  full  context  of  the 
observation  and  related  events. 

Members  of  the  LOSA  implementation  team  may  want  to  attend  LOSA  observer  training  at  another  organization  or 
attend  an  industry- sponsored  LOSA  conference  for  further  guidance  before  designing  and  customizing  the  training. 


TIPS: 

A-  Make  the  training  story-based.  Engage  trainees  with  practical  examples,  which  add  credibility  to  the  instructors 
and  training. 

Each  segment  of  LOSA  training  should  start  with  clearly  stated  learning  objectives.  Desired  learning  outcomes 
should  be  repeated  at  the  end  to  summarize  the  training. 

A-  Provide  handouts  for  trainees  to  follow  along  and  take  notes,  as  well  as  review  and  reference  after  the  training. 

A-  Frontline  employees  have  the  right  to  refuse  being  observed,  which  cannot  be  used  against  them. 

A-  Non-punitive  nature  of  the  LOSA  program:  Frontline  employees  should  be  informed  on  how  their  performance 
can  be  enhanced  instead  of  being  criticized  and  punished. 

Keep  anonymous  data  collected  and  information  obtained  from  each  LOSA  observation. 

A-  Document  and  share  lessons  learned  from  other  LOSA  programs  and  implementations,  so  the  new 
implementation  can  be  informed  and  prepared. 

^  Direct  LOSA  observers  to  external  resources,  e.g.,  the  project  website. 

KEY  REMINDERS: 

A-  Watch  out  for  those  things  that  may  kill  the  program  instantly: 

A-  Reporting  threats  and  errors  that  are  not  imminent  to  the  aircraft  or  personnel  safety  to  management,  which 
results  in  punitive  actions  in  various  formats  to  the  frontline  employees. 

A-  Set  up  and/or  run  LOSA  that  is  too  similar  to  other  existing  safety  programs  without  unique  safety  or 
financial  benefits;  failure  to  address  the  incorrect  perception  of  LOSA  as  an  existing  safety  program  in  a 
different  package. 

A-  Provide  LOSA  observers  with  written  SOP  for  doing  LOSA  observations. 

A-  LOSA  is  observation,  not  investigation  or  analysis 

FOR  ADDITIONAL  READING: 

A-  Maintenance  &  Ramp  LOSA  training  materials  and  templates  are  available  at  the  “Training”  section  of  the 
project  website:  www.MRLOSA.com 

A-  Bloom,  B.  (1956).  Bloom’s  Taxonomy  of  Learning  Domains  www.nwlink.com/~donclark/hrd/bloom.html 
A-  Types  of  Questions  based  on  Bloom’s  Taxonomy: 

www2.honolulu.hawaii.edu/facdev/guidebk/teachtip/questype.htm 
A-  Clarke,  D.  (2009)  Transfer  of  Learning:  A  brief  and  clear  introduction  -  including  an  explanation  of  why  transfer 
during  a  training  event  is  important. 
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www.nwlink.com/~donclark/hrd/learning/transfer.html 
^  FAA  Human  Factors:  Training  Transfer  Online  Reference: 

www.hf.  faa.  go  v/webtraining/T  raining/T  raining023  .htm 
^  How  to  Determine  Sample  Size:  http://edis.ifas.ufl.edu/pd006 
^  Online  Sample  Size  Calculator: 

A-  Creative  Research  System:  www.surveysystem.com/sscalc.htm 
^  Raosoft.com:  www.raosoft.com/samplesize.html 

A-  Phillips,  J.  J.  (1991).  Handbook  of  Training  Evaluation  and  Measurement  Methods,  Second  Edition.  Houston, 
TX:  Gulf  Publishing  Company. 

A-  Sierra,  K.  &  Russell,  D.  (2005).  Ten  Tips  for  New  Trainers/Teachers. 

http://headrush.typepad.eom/creating_passionate_users/2005/07/ten_tips_for_ne.html 
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IV.  Active  Implementation  Phase 
Step  7;  Conduct  LOSA  Observations 
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7.  Collect  Data 

> 

r 

Figure  12.  Step  7:  Collect  Data 

A  LOSA  cannot  succeed  without  the  full  and  candid  cooperation  of  the  frontline  employees,  and  there  can  be  no 
cooperation  without  trust.  In  addition  to  LOSA  awareness  training,  frontline  employees  should  be  informed  in  advance 
about  the  purpose  and  planned  implementation  of  a  LOSA.  A  letter  co-signed  by  credible  representatives  of  both 
management  and  labor  to  frontline  employees  is  strongly  recommended  in  order  to  assure  them  of  the  anonymity  and  non¬ 
jeopardy  status  of  LOSA  data.  Such  a  letter  should  also  include  a  disclaimer  giving  all  technicians  or  ramp  employees  the 
choice  of  declining  an  observation  at  their  discretion.  Only  by  building  in  these  guarantees  and  safeguards  will  the 
frontline  employees  feel  sufficiently  comfortable  to  act  normally  in  the  presence  of  a  LOSA  observer.  A  final  assurance 
should  be  an  in-house  publication  of  a  summary  of  LOSA  results  along  with  an  outline  of  initial  actions  and  proposed 
changes. 

Plan  a  reasonable  number  of  observations  per  observer  per  day  to  allow  sufficient  time  to  complete  the  observation 
coding  and  write  detailed  comments.  Scheduling  observers  across  stations  and  positions,  regardless  of  their  specialty, 
encourages  a  more  general,  cross-company  perspective  of  frontline  employee  performance.  Build  some  flexibility  into  the 
schedule  to  allow  for  the  unexpected.  Finally,  do  not  let  the  observations  continue  indefinitely — schedule  a  set  of 
observations  within  a  1-  to  3-  month  period,  if  possible.  The  data  needs  to  be  assessed  and  actions  implemented  in  a  timely 
fashion.  This  is  not  to  preclude  using  LOSA  observations  as  part  of  the  overall  SMS  set  of  tools  and  conducting  if  and 
when  needed  in  your  operations. 

Carrying  out  an  observation  is  a  skill  that  can  be  learned  and  enhanced  through  practice.  This  section  covers  the 
following  topics  for  both  Maintenance  LOSA  and  Ramp  LOSA: 

•  Observation  team 

•  Guidelines  for  how  to  conduct  the  LOSA  observations 

•  Stages  of  LOSA  observations 

•  Checklist  for  LOSA  observations 

Maintenance  LOSA 

The  maintenance  LOSA  observation  team 

For  Maintenance  LOSA  observations,  we  recommend  one  observer  per  task.  However,  for  complex  tasks,  like  an 
engine  change  involving  several  maintenance  technicians,  more  than  one  observer  may  be  needed.  The  one-person  team 
certainly  has  many  advantages.  First,  it  is  less  intrusive  and  threatening  to  the  maintenance  technician(s)  being  observed. 
Second,  due  to  the  nature  of  some  maintenance  tasks,  there  is  not  much  room  for  more  than  one  person  other  than  the 
working  technician  to  stand  and  move  around  (e.g.,  cockpit,  engine  cowling).  Third,  viewing  from  a  distance  may  not  be 
possible  (e.g.,  obstructed  view  or  cannot  see  clearly  until  up  close)  or  optimal  (e.g.,  poor  lighting  or  weather). 

Two  or  more  people  conducting  observations  of  a  typical  maintenance  task  could  make  the  maintenance  technician  feel 
uncomfortable.  The  key  to  a  LOSA  observation  is  that  you  want  the  people  being  observed  to  quickly  forget  you  are  there. 
However,  a  two-person  team,  including  a  union  observer,  may  be  helpful  at  the  initial  launch  stage  of  a  LOSA  program  at 
unionized  maintenance  organizations.  The  union  observer’s  presence  is  to  let  the  maintenance  technicians  know  that  the 
union  supports  the  LOSA  program,  which  helps  lower  their  apprehension,  and  helps  ensure  that  they  will  behave  as 
normally  and  as  naturally  as  possible. 
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The  observers  who  conduct  LOS  A  observations  should  be  carefully  selected  and  properly  trained  in  accordance  with 
the  training  guidelines  provided  in  this  document.  Additional  training  material  is  available  via  the  project  website 
(www.MRLOSA.com)  and  should  be  dovetailed  into  the  material  contained  within  this  guideline  for  training  staff. 

After  the  initial  introduction,  during  the  field  observation,  observers  must  multi-task.  First,  they  must  observe  how  the 
technician  conducts  a  given  task,  if  he  follows  the  SOPs  or  not,  what  the  observable  threats  are,  if  and  how  those  threats 
are  managed,  what  the  observable  errors  are,  if  and  how  those  errors  are  managed,  and  what  the  error  outcomes  are.  The 
observers  must  refer  to  the  observation  forms  from  time  to  time  to  ensure  all  observation  items  are  captured.  Second,  the 
observer  should  mark  in  the  observation  forms  coding  and  notes  periodically.  Third,  the  observer  should  take  the  “fly-on- 
the-wall”  approach  to  be  distant  from  the  maintenance  task  being  performed,  and  never  intervene  unless  the  aircraft  will 
not  be  airworthy  following  the  task,  there  is  imminent  personal  injury  danger  to  the  maintenance  technician(s),  the 
equipment,  or  the  aircraft  airworthiness.  Sometimes,  maintenance  technician(s)  being  observed  may  initiate  conversations 
trying  to  understand  more  about  the  LOSA  program  or  just  to  be  friendly.  However,  the  observer  should  politely  re¬ 
emphasize  it  is  important  not  to  discuss  the  program  specifics  or  engage  in  casual  conversations  during  the  observations, 
but  carry  out  the  task  as  if  no  other  person  were  there.  Only  upon  completion  of  an  observation  may  the  observer  discuss 
with  the  technician(s)  if  there  are  any  questions  or  suggestions.  However,  any  suggestions  to  the  technicians  should  be 
made  using  a  strictly  peer-to-peer,  friendly  and  constructive  tone. 


Guidelines  for  the  maintenance  LOSA  observation 

Field  observation  is  a  common  social  science  research  method  to  study  people  and  phenomena  within  natural  settings. 
Field  observation  studies  often  vary  in  three  ways:  the  amount  of  structure  imposed  on  the  observations,  whether  or  not 
the  participants  are  aware  they  are  being  observed,  and  the  degree  to  which  the  observer  interacts  with  the  people  in  the 
setting. 

The  LOSA  program  adopts  one  type  of  field  observation  method  called  “direct  (reactive)  observation,”  which  means 
that  people  know  that  you  are  observing  them.  One  major  problem  with  this  method  is  that  behavior  may  change  because 
of  the  presence  of  the  observer.  Individuals  may  change  their  actions,  rather  than  behave  normally.  Although  people  may 
alter  their  behavior  when  they  know  they  are  being  observed,  researchers  often  find  that,  over  prolonged  periods,  people 
tend  to  forget  or  ignore  that  they  are  being  observed  and  revert  to  their  normal  behavior.  Thus,  an  observational  study  will 
eventually  observe  the  natural  behavior. 

LOSA  observers  are  trained  to  reveal  the  purpose  of  the  observation  but  “lay  back”  and  observe  from  a  distance  (not 
necessarily  physical),  with  as  little  distraction  as  possible.  However,  because  the  LOSA  program  uses  peer-to-peer 
observations,  the  observers  are  emotionally  involved  with  the  people  in  the  work  setting,  although  they  can  be  trained  to 
minimize  that  involvement.  The  LOSA  observers  should  only  ask  questions  of  people  in  the  work  setting  to  gain  a  fuller 
understanding  upon  completion  of  the  LOSA  observation. 

Stages  of  the  maintenance  LOSA  observations 

Stage  1:  Prepare  for  the  Field  Observation 

You  may  be  provided  with  detailed  instructions  on  how  to  proceed  with  the  following  four  steps,  or  you  may  be  asked 
to  work  with  the  LOSA  program  manager  and  other  observers  to  make  a  cohesive  plan  together. 

l.a  Do  your  homework 

The  LOSA  program  manager  may  assign  you  a  specific  maintenance  task  to  observe  across  multiple  shifts  over  several 
days  or  weeks,  or  several  maintenance  tasks  over  the  same  period  of  time.  Sometimes,  as  the  subject  matter  expert,  you 
may  be  consulted  to  suggest  the  most  problematic  areas  o  r  tasks  to  be  the  focus  of  a  LOS  A  observation,  the  number  of 
maintenance  tasks  to  obse  rve,  and  when  or  where  to  carry  out  those  observations.  Before  each  observation,  determine 
what  you  a  re  trying  to  learn  (e.g.,  the  most  complained  about  and  least  pap  erwork-compliant  maintenance  tasks),  and 
develop  a  clear  m  ental  image  of  the  observation  objectives.  This  will  help  you  develop  your  observation  plan  and  select 
what  task  (e.g.,  lock  out  and  tag  out  procedure)  and  whom  or  where  to  observe. 

l.b  Plan  for  your  field  observations 

For  example,  suppose  you  are  to  conduct  observations  related  to  how  technicians  currently  lock  out  and  tag  out  systems 
or  equipment  prior  to  performing  a  m  aintenance  task.  Depending  on  your  focus,  you  may  have  a  broad  scope  (e.g.,  do 
technicians  run  into  any  problems  with  lock  out  and  tag  out  procedures  in  place?)  or  a  narrow  scope  (e.g.,  particular  tag 
and  paperwork  people  that  use  to  lock  out  and  tag  out). 

Whatever  your  focus,  you  need  to  decide  which  segment  of  the  maintenance  task  you  need  to  observe  (e.g.,  planning, 
removal,  servicing)  and  print  out  the  corresponding  sections  of  the  maintenance  LOSA  observation  forms  and  review  them 
carefully.  Print  out  and  review  demographic,  additional  threats  or  errors,  and  acronyms  sections  of  the  forms,  as  well  as 
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threat  cod  es.  It  is  also  i  mportant  to  rev  iew  an  y  writte  n  stand  ard  procedures,  m  anuals,  or  job  card  s  for  th  e  given 
maintenance  task. 

Try  to  plan  with  an  open  mind  and  empathy  for  the  technician(s).  It  is  important  to  document  the  management  or 
mismanagement  of  threats  and  errors,  but  even  more  importantly,  you  should  try  to  understand  the  context  in  which  such 
actions  arose.  Your  decisions  on  whether  the  technician  is  doing  the  task  correctly  should  be  based  on  what  is  required  by 
the  maintenance  documentation  and  not  based  on  how  you  would  perform  the  task  or  ho  w  you  always  thought  it  should 
have  been  modified  or  improved.  During  the  planning  stage,  you  should  not  look  at  yourself  as  a  qualified  maintenance 
technician  at  work  but,  rather,  as  an  objective  observer.  In  other  words,  let  your  knowledge,  skill,  and  experience  guide 
you  in  the  interpretation  of  what  you  will  observe,  but  do  not  let  any  preconception  take  over  the  observation. 

l.c  Select  sites 

The  sites  should  be  at  places  and  tim  es  that  are  relevant  for  your  observation  objective.  You  should  be  able  to  observe 
without  t  oo  m  uch  disturbance  and  without  making  t  he  t  echnicians  at  wo  rk  feel  sel  f-conscious.  Th  e  LOS  A  program 
manager  s  hould  help  y  ou  obtain  t  he  permission  o  f  m  anagement  (an  d  1  abor  or  employee  gr  oup  1  eadership)  t  o  ac  cess 
selected  sites. 

l.d  Check  into  the  sites 

When  you  arrive  at  the  sites,  you  should  check  in  and  obtain  an  acknowledgment  from  the  supervisor  and  lead  that  you 
will  spend  a  few  hours  walking  around,  observing  the  premises,  and  briefly  talking  to  people  afterwards.  However,  expect 
questions,  especially  if  you  are  in  an  area  you  will  not  normally  be  in  and  walking  around  with  a  clipboard  taking  notes. 

Stage  2:  Conduct  Field  Observation 

Approach  the  technician(s)  you  plan  to  observe,  greet  them,  introduce  yourself,  and  briefly  explain  what  you  are  trying 
to  do.  They  should  already  have  been  briefed  on  the  LOS  A  process  by  this  time.  Ask  for  their  permission  to  conduct  an 
LOS  A  observation.  Walk  away  so  that  the  technicians  can  discuss  it,  and  accept  without  question  any  technician’s 
decision  to  deny  the  opportunity  to  be  observed.  It  is  recommended  to  carry  a  copy  of  the  letter  of  endorsement  jointly 
signed  by  management  and  the  labor  or  employee  group  to  show  to  any  interested  frontline  employee.  Thank  the 
technician(s)  if  they  agree  to  be  observed.  Explain  that  you  may  ask  some  demographic  questions  after  the  observation. 
You  may  also  ask  some  clarifying  questions  about  why  somebody  did  something  when  the  threats  were  not  observable. 
However,  the  technician  does  not  have  to  answer  any  questions  he  does  not  feel  comfortable  in  answering.  There  are 
several  options  and  a  range  of  practices  for  obtaining  feedback,  e.g.,  seeking  immediate  feedback  after  each  observation, 
or  seeking  cumulative  feedback  after  a  group  of  observations.  The  critical  part  is  to  involve  frontline  employees  in 
discussing  safety  in  a  “peer-to-peer”  atmosphere.  Stress  that  the  post-observation  discussion  is  not  any  form  of 
investigation;  its  sole  purpose  is  to  provide  an  opportunity  for  two-way  communication  to  gain  demographic  data  and  to 
clarify  any  issues  either  you  or  the  technician(s)  may  have. 

Keep  in  mind  your  observation  objectives.  Keep  an  open  mind  and  try  to  get  a  good  idea  or  scheme  of  the  actions  that 
are  taking  place.  If  there  are  many  people  participating,  make  notes  of  what  each  one  does  and  how  they  interact.  Try  to 
find  out  whether  the  actions  occurring  are  normal  or  exceptional. 

Remember,  you  may  not  see  the  threats  that  lead  to  the  error(s).  Never  speculate  or  guess  what  the  threats  were  during 
your  observation.  You  must  see,  hear,  smell,  or  touch  a  threat  to  confirm  it.  The  safety  information  derived  from  LOSA 
observation  data  is  only  as  valid  as  the  data  we  collect.  We  want  to  avoid  “garbage-in,  garbage-out.” 

You  should  become  very  familiar  with  the  observation  forms  as  you  use  them  over  time.  Use  the  forms  as  a  ch  ecklist 
and  refer  to  them  from  time  to  time.  The  f orms  are  for  a  memory  aid.  You  may  get  distracted  or  interrupted  or  become 
tired  or  bored,  so  do  not  solely  depend  on  your  memory  during  the  observation. 

Go  t  hrough  t  he  f  orms  befo  re  y  ou  c  onclude  t  he  observation  to  see  if  the  re  are  a  ny  fields  that  were  le  ft  blank 
unintentionally.  If  there  are  items  that  you  did  not  observe,  were  safe,  or  not  applicable,  mark  them  accordingly.  You  do 
not  have  to  fill  out  all  the  fields  (e.g.,  threat  codes)  if  you  do  not  observe  the  threat  or  activity;  however,  it  is  essential  that 
all  the  mandatory  fields  in  the  demographics  form  are  filled  out. 

It  is  always  helpful  to  docu  ment  some  notes  on  t  he  context  of  threats  and  errors.  The  notes  will  help  you  recall  and 
organize  your  observation  afterwards.  They  will  be  useful  to  the  analysts  to  document  your  observations  in  the  database 
and  generate  detailed  reports.  Write  as  many  notes  as  you  can.  Take  breaks  to  review  and  organize  them. 

Document  your  observation  number.  Check  the  “Did  Not  Observe”  box  if  you  did  not  observe  a  certain  section.  Write 
down  additional  threats  and  errors  in  the  corresponding  form.  Make  sure  your  handwriting  is  legible. 

Keep  an  ope  n  mind,  eve  n  if  you  feel  pe  ople  are  doing  things  wron  g.  Try  to  un  derstand  why  th  ey  are  actin  g  in  a 
particular  way  .  What  kn  owledge  or  e  xperience  are  t  hey  dr  awing  on  ?  What  fact  ors  in  their  envi  ronment  enabl  e  or 
encourage  them  to  act  in  this  way?  What  broader  routine  or  purpose  is  this  action  part  of? 

Upon  completion  of  your  observation,  make  sure  y  ou  have  addressed  all  the  questions  you  want  answered.  Ask  the 
people  on  your  observation  team  (if  any)  if  they  have  anything  they  would  like  to  add. 
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Remember  to  thank  the  people  you  have  observed  for  their  time.  Tell  them  how  to  get  in  touch  with  you  if  they  have 
any  questions  and  leave  them  your  contact  information  (if  they  do  not  already  have  it).  Tell  them  what  the  next  steps  will 
be  and  how  observational  data  will  be  used. 

After  your  observation,  go  through  your  coding  and  notes  in  the  observation  forms,  and  write  up  additional  notes  as 
soon  as  you  can.  Remember  to  clearly  label  all  material  related  to  each  LOSA  observation. 

Stage  3.  Organize/Summarize  Field  Observations 

Review  your  LOSA  observation  notes  and  finish  incomplete  coding  as  soon  as  practical.  Because  memory  of  details, 
dialogue,  and  the  sequence  of  events  fades  fast,  field  no  tes  are  written  as  clo  se  in  ti  me  as  feasi  ble  to  th  e  observations. 
When  unobtrusively  observing  from  a  distance,  notes  are  written  on  the  spot.  There  are  always  tradeoffs,  as  observers  may 
risk  missing  something  noteworthy  while  writing. 

Key  principles  for  the  maintenance  LOSA  observation 

There  are  some  key  principles  to  guide  LOSA  observations.  These  include: 

1 .  Before  each  observation,  de  termine  what  you  are  t  rying  to  learn  and  de  velop  a  c  lear  mental  image  of  the 
observation  objectives.  Print  out  and  review  all  relevant  observation  forms  and  supporting  documents. 

2.  Obtain  and  review  any  written  standard  procedures,  manuals,  job  cards,  etc.  for  the  given  maintenance  task  you  are 
going  to  observe. 

3.  Take  the  “fly-on-the-wall”  approach,  and  distance  yourself  from  the  maintenance  task  being  observed. 

4.  Never  intervene  unless  there  is  an  imminent  danger  to  the  maintenance  technician(s),  the  aircraft,  the  equipment,  or 
if  the  aircraft  is  going  to  be  left  in  a  non  -airworthy  condition.  If  you  see  a  t  echnician  carry  out  a  pa  rt  of  a  t  ask 
incorrectly,  and  that  error  will  make  the  aircraft  non-airworthy,  then  you  need  to  decide  when  to  int  ervene.  You 
want  to  give  the  technician  time  to  recover  from  his  error.  On  the  other  hand,  if  the  te  chnician  does  not  seem  to 
become  aware  of  t  he  error,  you  do  not  want  to  wait  for  him  to  finish  a  task  and  t  hen  have  him  backtrack  for  3  0 
minutes  on  the  task  to  get  to  the  error.  Some  judgment  on  your  part  will  be  required  to  decide  when  to  intervene. 

5.  During  LOSA  observations,  if  maintenance  technician(s)  being  observed  initiate  casual  conversations  with  you,  you 
should  politely  re-emphasize  it  is  important  to  not  engage  in  discussion  but  carry  out  the  task  as  if  no  one  else  were 
there. 

6.  Document  the  management  or  mismanagement  of  threats  and  errors,  as  well  as  the  cont  ext  in  which  such  actions 
arise. 

7.  To  ensure  the  quality  of  data,  never  speculate  or  guess  about  what  the  threats  were  or  whether  other  activities  that 
you  did  not  observe  directly  during  your  observation.  You  must  be  abl  e  to  see,  hea  r,  smell,  or  t  ouch  a  t  hreat  or 
activity  to  confirm  it. 

8.  Use  the  LOSA  observation  forms  as  a  checklist  and  refer  to  them  from  time  to  time. 

9.  Only  u  pon  c  ompletion  of  a  n  o  bservation,  y  ou  m  ay  di  scuss  wi  th  t  echnician(s)  i  f  t  here  a  re  any  questions  or 
comments.  A  LOSA  ob  servation  is  no  t  an  inv  estigation.  Th  e  so  le  pu  rpose  of  po  st-observation  d  iscussion  is  to 
gather  additional  demographic  information  to  help  you  better  understand  or  clarify  things  observed  earlier. 

10.  Upon  completion  of  your  observation,  go  through  your  coding  and  notes  in  the  observation  forms,  and  write  up 
additional  notes  as  soon  as  you  can. 

Checklist  for  maintenance  LOSA  observations 

Remember  to  prepare  the  following: 

□  Permission  to  observe  in  the  location  that  you  have  selected  (and  separate  permission  for  e.g.,  photographing) 

□  A  plan  of  what  and  how  you  are  going  to  observe  and  which  people  you  should  talk  to 

□  An  open  mind  -  try  to  recognize  and  avoid  preconceptions 

□  Pen  and  notepad  with  LOSA  forms  and  supporting  documents  and  sufficient  paper  for  taking  notes 

□  Camera  or  audio  recorder  (if  they  are  a  part  of  the  LOSA  program  protocol  at  your  organization) 

□  Your  business  card  or  the  LOSA  program  manager’s  card  or  contact  information  so  the  people  you  have  talked  to  can 

get  in  touch  with  you  if  they  have  questions  or  further  information 

□  Personal  safety  protection  gear  (e.g.,  earplugs,  goggles,  hard  toe  shoes,  reflective  safety  vest) 

□  Maintenance  technician  tools  (e.g.,  flashlight) 

Remember  to  do  the  following: 

□  Arrive  on  time  and  be  well  prepared 

□  Introduce  yourself  and  explain  why  you  are  there 

□  Ask  permission  to  do  the  LOSA  observation 
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□  Explain  briefly  how  information  collected  will  be  used  and  toward  what  goal 

□  Do  not  disturb  the  people  you  are  observing 

□  Look  and  listen  closely  and  take  notes 

□  Complement  your  observations  with  a  short  discussion  only  upon  completion  of  your  observation 

□  Ensure  that  the  information  will  be  used  anonymously  and  kept  safe 

□  Write  up  your  notes  and  finish  up  coding  right  after  each  observation 

□  Make  sure  that  all  materials  collected  are  labelled  properly 

□  Dress  properly  and  wear  all  required  safety  and  security  badges  visibly 


Ramp  LOSA 

The  ramp  LOSA  observation  team 

According  to  the  specific  needs  of  your  organization  and  a  particular  LOSA  implementation,  your  company  LOSA 
implementation  team  will  decide  whether  your  observations  should  be  fleet  specific,  specific  to  a  particular  phase  of  the 
ramp  operations,  specific  to  an  airport,  or  specific  to  an  area  of  an  airport.  For  each  turn-around,  a  team  of  two  or  three 
observers  is  recommended  depending  on  the  type  of  aircraft  (e.g.,  wide  body,  narrow  body,  turboprops,  and  regional  jets) 
and  the  activities  that  you  will  be  observing  (e.g.,  you  may  decide  not  to  observe  the  cleaning  crew).  The  team  can  choose 
to  divide  observations  during  a  turn-around  by  the  areas  of  the  aircraft  (e.g.,  forward  above  the  wings,  aft  below  the 
wings)  or  by  the  specific  tasks  (e.g.,  lavatory/potable  water  service,  catering,  fueling),  or  a  combination  of  the  two  (e.g., 
left  of  the  aircraft  during  downloading,  boarding  bridge  during  arrival).  The  team  members  need  to  discuss  and  visualize  a 
plan  ahead  of  time  regarding  where  each  observer  positions  himself,  and  be  open-minded  and  flexible  during  the 
observation.  You  may  not  find  the  best  approach  to  coordinate  observations  right  away,  but  it  may  take  time  through  trial 
and  error  after  three  or  four  turn-arounds.  To  make  the  observation  procedures  more  manageable,  we  recommend  dividing 
the  14-section  ramp  LOSA  observation  forms  into  multiple  sections;  each  observer  is  responsible  for  three  to  four  sections 
(in  another  words,  three  to  four  ramp  activities).  However,  make  sure  each  observer  has  a  copy  of  the  demographic  form, 
additional  threat  and  error  forms,  and  the  acronym  list.  The  bottom  line  is  when  there  are  multiple  observers,  based  on  a 
previously  agreed-upon  coordination  plan,  each  observer  should  observe  and  record  as  much  as  possible.  You  want  team 
members  to  overlap  in  their  observations  and  notes  to  capture  a  comprehensive  picture. 

The  observation  team  should  try  to  be  as  least  intrusive  and  threatening  as  possible  to  the  ramp  employees  when  they 
divide  tasks  and  position  themselves.  The  key  to  field  observation  is  that  you  want  the  people  being  observed  to  quickly 
forget  you  are  even  there.  A  union  observer  may  be  helpful  at  the  initial  launch  stage  of  a  LOSA  program  at  unionized 
airlines  and  ground  operators.  The  union  observer’s  presence  is  to  let  the  ramp  employees  know  that  the  union  supports 
the  LOSA  program,  help  lower  their  apprehension,  and  help  ensure  that  they  will  behave  as  normally  and  as  naturally  as 
possible. 

A  ramp  LOSA  observation  can  sometimes  be  a  very  demanding  task  because  of  the  quick  turn-around  between  the 
arrival  and  departure  of  a  flight.  To  maximally  assist  the  observation,  the  ramp  LOSA  observation  forms  were  designed  to 
use  the  checklist  style  for  easy  tracking  plus  some  spaces  for  additional  comments. 

The  observers  who  conduct  LOSA  observations  should  be  carefully  selected  and  properly  trained  in  accordance  with 
the  training  guidelines  provided  in  this  guideline.  Additional  training  material  is  available  via  the  project  website 
(www.MRLOSA.com)  and  should  be  dovetailed  into  the  material  contained  within  this  guideline  for  training  staff. 

After  the  initial  introduction,  during  the  field  observation,  each  of  the  observers  has  to  multi-task.  First,  he  has  to  use  all 
senses  to  see,  hear,  smell,  and  touch  how  the  ramp  employee(s)  conducts  a  given  task,  if  the  standard  operating  procedures 
have  been  followed,  what  the  observable  threats  are,  if  and  how  those  threats  are  managed,  what  the  observable  errors  are, 
if  and  how  those  errors  are  managed,  and  what  the  error  outcomes  are.  The  observer  has  to  refer  to  the  observation  forms 
from  time  to  time  to  ensure  all  observable  items  are  captured.  Second,  the  observer  should  mark  in  the  observation  forms 
with  threat  or  error  codes  and  notes  periodically.  Third,  the  observer  should  take  the  “fly-on-the-wall”  approach  and 
distance  himself  from  the  ramp  task  being  observed,  and  never  intervene  unless  there  is  an  imminent  danger  to  the  ramp 
employees(s),  the  aircraft,  or  the  equipment.  Sometimes,  ramp  employee(s)  being  observed  may  initiate  conversations 
trying  to  understand  more  about  the  LOSA  program  or  just  to  be  friendly.  The  observer  should  politely  re-emphasize  it  is 
important  not  to  engage  in  discussion  but  carry  out  the  task  as  if  no  other  person  were  there.  Only  upon  completion  of  an 
observation,  the  observer  may  discuss  with  ramp  employee(s)  if  there  are  any  questions  or  suggestions.  However,  any 
suggestions  to  the  ramp  employee(s)  should  be  made  in  a  strictly  peer-to-peer,  friendly,  and  constructive  tone. 
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Guidelines  for  the  ramp  LOSA  observation 

Field  observation  is  a  common  social  science  research  method  to  study  people  and  phenomena  within  natural  settings. 
Field  observation  studies  often  vary  in  three  ways:  the  amount  of  structure  imposed  on  the  observations,  whether  or  not 
the  participants  are  aware  they  are  being  observed,  and  the  degree  to  which  the  observer  interacts  with  the  people  in  the 
setting. 

The  LOSA  program  adopts  a  type  of  field  observation  method  known  as  “direct  (reactive)  observation,”  which  means 
that  people  know  that  you  are  observing  them.  One  major  problem  with  this  method  is  that  behavior  may  change  because 
of  the  presence  of  the  observer.  Individuals  may  change  their  actions  rather  than  behaving  normally.  Although  people  may 
alter  their  behavior  when  they  know  they  are  being  observed,  researchers  often  find  that,  over  prolonged  periods,  people 
tend  to  forget  or  ignore  they  are  being  observed  and  revert  to  their  normal  behavior. 

Ramp  LOSA  observation  adopts  a  highly  structured  approach  and  use  LOSA  observation  forms  with  a  predefined  set 
of  categories  of  behaviors.  Especially  considering  the  quick  turn-around  between  the  arrival  and  departure  of  a  flight,  the 
ramp  LOSA  observation  forms  were  written  as  multiple  checklists  with  some  spaces  for  additional  comments. 

LOSA  observers  are  trained  not  to  reveal  their  purpose  of  the  observation  but  “lay  back”  and  observe  from  a  distance 
(not  necessarily  physical  distance),  with  as  little  distraction  as  possible.  However,  because  the  LOSA  program  utilizes 
peer-to-peer  observations,  the  observers  are  known  by  the  people  in  the  setting,  although  the  observer  can  be  trained  to 
minimize  that  involvement.  The  LOSA  observer  should  only  ask  questions  of  people  in  the  work  setting  to  gain  a  fuller 
understanding  upon  completion  of  the  LOSA  observation. 

Stages  of  the  ramp  LOSA  observations 

Stage  1:  Prepare  for  the  Field  Observation 

You  may  be  provided  with  detailed  instruction  on  how  to  proceed  with  the  following  four  steps,  or  you  may  be  asked  to 
work  with  the  LOSA  program  manager  and  other  observers  to  make  a  cohesive  plan  together. 

l.a  Do  your  homework 

The  LOSA  program  manager  may  assign  your  team  a  specific  flight  to  observe  across  several  days  or  weeks,  or  several 
flights  over  t  he  sam  e  peri  od  o  f  t  ime.  Som  etimes  y  ou,  as  t  he  s  ubject  matter  expe  rt,  m  ay  be  c  onsulted  for  t  he  most 
problematic  areas  or  tasks  to  be  the  focus  of  a  LOSA  implementation  round,  the  number  of  flights  to  observe,  and  when  or 
where  to  carry  out  those  observations.  For  each  observation,  plan  out  in  adva  nee  what  you  are  t  rying  to  learn  (e  .g.,  the 
most  complained-about  and  least  paperwork-complied  ramp  tasks),  and  develop  a  clear  mental  image  of  the  observation 
objectives.  This  will  h  elp  you  develop  your  observation  plan  and  select  wh at  (e.g.,  hearing  protection  misuse,  including 
not  use)  and  whom  or  where  to  observe. 

l.b  Plan  for  your  field  observations 

Depending  on  your  focus,  you  can  have  a  broad  scope  or  a  more  narrow  scope.  For  example,  you  may  want  to  conduct 
observations  that  are  specific  to  how  ramp  employees  currently  use  and  misuse  Personal  Protective  Equipment  (PPE).  In 
this  LOSA,  a  broad  scope  can  be  how  people  misuse  hearing  protection  such  as  earplugs  on  the  ramp,  or  a  narrow  scope 
can  include  misuse  of  hearing  protection  such  as  earplugs  due  to  communication  needs. 

Whatever  your  focus,  you  need  to  decide  which  segment  of  ramp  tasks  to  observe  (e.g.,  arrival,  downloading,  push- 
back)  and  print  out  the  corresponding  sections  of  the  ramp  LOSA  observation  forms  and  review  them  carefully.  Print  out 
and  review  demographic,  additional  threats  or  errors,  and  acronyms  sections  of  the  forms,  as  well  as  the  threat  a  nd  error 
codes.  It  is  also  important  to  review  any  written  standard  procedures  or  manuals  for  the  given  ramp  tasks.  Because  ramp 
LOSA  observations  take  place  outdoors,  we  recommend  you  weatherproof  your  threat  and  error  code  cards  by  laminating 
them.  Color-coding  different  groups  of  the  threat  and  error  codes  will  aid  in  your  visual  search.  In  addition,  you  may  want 
to  carry  a  pamphlet-sized  one  (folding  an  8”xll”  paper  into  half)  out  on  the  ramp,  and  keep  a  regular-sized  (e.g.,  8”xl  1”, 
also  color-coded  and  laminated)  for  office  use.  Instead  of  printing  out  ramp  LOSA  observation  forms  on  8”xl  1”  paper  and 
clip  them  on  a  clip  board,  some  observers  may  find  it  more  convenient  to  customize  the  size  of  ramp  LOSA  observation 
forms  by  reducing  them  and  making  a  booklet. 

Try  to  plan  with  an  open  mind  and  empathy  for  the  ramp  employees.  It  is  important  to  document  the  management  and 
mismanagement  of  t  hreats  and  errors,  but  even  more  important,  to  try  to  understand  the  context  in  which  such  actions 
arise.  Y our  plan  should  not  include  how  you  would  perform  the  task  or  ho  w  you  always  thought  it  should  have  been 
modified  or  improved.  During  the  planning  stage,  you  need  consider  yourself  not  as  a  qualified  ramp  employee  at  work 
but  as  an  objective  ob  server.  In  ano  ther  words,  let  you  r  knowledge,  sk  ills,  an  d  ex  perience  faci  litate  yo  u  in  the 
interpretation  of  what  you  will  observe,  but  do  not  let  them  do  the  observation  for  you. 

l.c  Select  sites 

The  sites  should  be  places  and  times  that  are  relevant  for  your  observation  objective.  They  should  also  be  available  and 
you  should  be  able  to  observe  without  too  much  disturbance  and  without  making  the  ramp  employees  feel  self-conscious. 
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The  LOS  A  program  manager  should  help  you  obtain  the  permission  of  management  (and  union  leadership)  to  access 
selected  sites. 

l.d  Check  into  the  sites 

When  you  arrive  at  the  sites,  you  should  check  in  and  obtain  acknowledgment  from  the  supervisor  and  lead  that  you 
will  spend  a  few  hours  walking  around,  observing  the  premises,  and  briefly  talking  to  people  afterwards.  However,  expect 
questions,  especially  if  you  are  in  an  area  you  would  not  normally  be  in  and  walking  around  with  a  clipboard  taking  notes. 

Stage  2:  Conduct  Field  Observation 

Approach  the  ramp  employee(s)  you  plan  to  observe,  greet,  and  briefly  explain  what  you  are  trying  to  accomplish,  and 
ask  for  their  permission  to  conduct  a  LOSA  observation.  Walk  away  so  that  the  ramp  employees  can  discuss  it,  and  accept 
without  question  any  employee’s  decision  to  deny  the  opportunity  to  be  observed.  It  is  recommended  to  carry  a  copy  of 
the  letter  of  endorsement  jointly  signed  by  management  and  the  labor  group  to  show  any  interested  frontline  employee. 
Thank  the  ramp  employee(s)  if  they  agree  to  be  observed.  Explain  that  you  may  ask  some  demographic  questions  after  the 
observation.  You  may  also  ask  some  questions  such  as  who  the  ramp  employee  was  talking  to  on  the  headset.  But  do  not 
ask  people  why  they  did  or  did  not  do  some  activity,  because  this  will  be  threatening  and  take  away  from  the  feeling  of 
anonymity.  Ramp  employees  do  not  have  to  answer  any  questions  they  do  not  feel  comfortable  to  answer.  There  are 
several  options  and  a  range  of  practices  for  obtaining  feedback,  e.g.,  seeking  immediate  feedback  after  each  observation  or 
seeking  cumulative  feedback  after  a  group  of  observations.  The  critical  part  is  to  involve  frontline  employees  in  discussing 
safety  in  a  “peer-to-peer”  atmosphere.  Emphasize  the  post-observation  discussion  is  not  any  form  of  investigation;  its  sole 
purpose  is  to  provide  an  opportunity  for  two-way  communication  to  gain  demographic  data  and  to  clarify  any  issues  either 
you  or  the  ramp  employee(s)  may  have. 

Keep  in  mind  your  observation  objectives.  Have  an  open  mind  and  try  to  get  a  good  idea  or  scheme  of  the  actions  that 
are  taking  place.  If  there  are  many  people  participating,  make  notes  of  what  each  one  does  and  how  they  interact.  Try  to 
find  out  whether  the  actions  occurring  are  normal  or  exceptional. 

Remember,  you  m  ay  not  see  the  threats  t  hat  lead  to  the  error(s).  Never  speculate  about  what  threats  caused  an  error 
during  your  observation  unless  you  physically  sense  the  threat  yourself.  The  safety  information  we  derive  from  LOSA 
observation  data  is  only  as  valid  and  good  as  the  data  we  collect.  We  want  to  avoid  “garbage-in,  garbage-out.” 

You  should  become  very  familiar  with  the  observation  forms  as  time  goes  by.  However,  use  the  forms  as  a  checkl  ist 
and  refer  to  th  em  from  time  to  time.  The  form  s  are  there  to  be  a  memory  aid.  You  may  get  distracted,  interrupted,  or 
become  tired  or  bored,  so  do  not  solely  depend  on  your  memory  during  the  observation. 

What  is  t  he  true  relationship  between  threats  an  d  errors?  Threats  can  cause  errors  if  thr  eats  are  not  identified  and/or 
properly  managed.  For  example,  movable  stairways  may  cause  door  damage.  Errors  can  certainly  cause  additional  threats. 
For  example,  soda  bottles  left  on  Ground  Support  Equipment  (GSE)  can  become  a  Foreign  Object  Damage  (FOD)  threat. 

Go  through  the  forms  before  you  conclude  the  observation  to  see  if  there  are  any  fields  were  left  blank  un-intentionally. 
If  there  are  items  that  you  did  not  observe,  were  safe  or  not  applicable,  mark  them  accordingly.  However,  you  do  not  have 
to  fill  o  ut  all  t  he  fields  (e.g.,  th  reat  codes)  if  you  do  not  know  what  the  threat  was,  for  ex  ample.  Double  check  all  th  e 
mandatory  fields  in  the  demographics  form  are  filled  out. 

It  is  always  helpful  to  docu  ment  some  notes  on  t  he  context  of  threats  and  errors.  The  notes  will  help  you  recall  and 
organize  your  observation  afterwards.  They  will  be  useful  to  the  analysts  to  document  your  observations  in  the  database 
and  generate  detailed  reports.  Write  as  many  notes  as  you  can.  Take  breaks  to  review  and  organize  your  notes. 

Document  your  observation  number.  Check  the  “Did  Not  Observe”  box  if  you  did  not  observe  a  certain  section  of  the 
observation  form.  Write  on  additional  threats  and  errors.  Make  sure  your  handwriting  is  legible. 

On  a  piece  of  bla  nk  pa  per,  you  m  ay  want  to  m  ake  co  mments  and  write  a  dditional  notes  detailing  the  overall 
observation.  Y ou  may  want  to  draw  pictures  of  t  he  airplane  or  GSE  to  help  explain  a  point  (e.  g.,  which  tire  was  not 
chocked).  You  may  also  want  to  doc  ument  the  number  of  equipment  pieces  that  were  used  (e.g.,  two  loader  s,  one  foo  d 
truck,  and  one  lavatory  service). 

Do  not  skip  the  demographics  section  because  demographics  provide  rich  background  and  contextual  information.  As  a 
part  of  preplanning,  you  and  your  observation  teammates  may  want  to  discuss  how  to  collectively  gather  demographics 
prior  to,  during,  and  after  observations,  so  multiple  observers  observing  the  same  flight  will  not  repeat  entering  the  same 
flight  or  personnel  data. 

Keep  an  ope  n  mind,  eve  n  if  you  feel  pe  ople  are  doing  things  wron  g.  Try  to  un  derstand  why  th  ey  are  actin  g  in  a 
particular  way  .  What  kn  owledge  or  e  xperience  are  t  hey  dr  awing  on  ?  What  fact  ors  in  their  envi  ronment  enabl  e  or 
encourage  them  to  act  in  this  way?  What  broader  routine  or  purpose  is  this  action  part  of? 

Upon  completion  of  your  observation,  make  sure  you  have  addressed  all  the  questions  you  want  answered.  Ask  your 
observation  team  members  if  they  have  anything  they  would  like  to  add. 
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Remember  to  thank  the  people  you  observed  for  th  eir  time.  Tell  th  em  how  to  get  in  touch  with  you  if  th  ey  have  any 
questions  and  leave  them  your  or  the  LOSA  program  manager’s  contact  information  (if  they  do  not  already  have  it).  Tell 
them  what  the  next  steps  will  be  and  how  you  will  use  the  information  they  have  given  you. 

After  your  observation,  go  through  your  coding  and  notes  in  the  observation  forms,  and  write  up  additional  notes  as 
soon  as  you  can.  Remember  to  clearly  label  all  material  related  to  each  LOSA  observation. 

Please  note,  the  current  ramp  LOSA  program  and  observation  forms  do  not  cover  the  following  ramp  related  activities 
and  support  tasks:  bag  room  and  sorting,  cargo  warehouse,  mail  sorting,  quick  pack,  ticket  counter,  facility  maintenance, 
GSE  maintenance  and  repair  shops,  maintenance  hangars,  associated  shops,  etc. 

Stage  3.  Organize  and  Summarize  Field  Observations 

Review  your  LOSA  observati  on  notes,  and  finish  incomplete  coding  as  soon  as  practical.  Because  m  emory  of  details, 
dialogue,  and  the  sequence  of  events  fades  fast,  field  no  tes  are  written  as  clo  se  in  ti  me  as  feasi  ble  to  th  e  observations. 
When  unobtrusively  ob serving  from  a  d  istance,  notes  are  writte  n  on  the  spot.  There  are  al  ways  tradeoffs,  however,  as 
these  observers  risk  missing  something  noteworthy  while  writing. 

Key  principles  for  the  ramp  LOSA  observation 

There  are  some  key  principles  to  guide  LOSA  observations.  These  include: 

1 .  Before  eac  h  observation,  determine  what  you  a  re  trying  to  learn,  a  nd  de  velop  a  cl  ear  m  ental  image  of  t  he 
observation  objectives.  Print  out  and  review  all  relevant  observation  forms  and  supporting  documents. 

2.  Obtain  and  review  any  written  standard  procedures  for  the  given  ramp  task  you  are  going  to  observe. 

3.  Take  the  “fly-on-the-wall”  approach,  and  distance  yourself  from  the  ramp  task  being  observed. 

4.  Never  inte  rvene  unless  t  here  is  an  imminent  da  nger  t  o  the  ram  p  em  ployee(s)  at  work  or  the  airc  raft  or  the 
equipment. 

5.  During  LOSA  observations,  if  ramp  employee(s)  being  observed  initiate  casual  conversations  with  the  observer, 
you  sh  ould  politely  re-e  mphasize  it  is  i  mportant  n  ot  to  d  iscuss  th  e  LOSA  pro  cess  o  r  eng  age  in  casu  al 
conversations,  but  carry  out  the  task  as  if  no  other  person  were  there. 

6.  Document  the  management  and  mismanagement  of  threats  and  errors,  as  well  as  the  context  in  which  such  actions 
arise. 

7.  To  ensure  the  quality  of  data,  neve  r  speculate  about  the  threats  or  othe  r  information  during  your  observatio  n 
unless  you  see,  hear,  smell,  or  touch  to  confirm  them. 

8.  Use  the  LOSA  observation  forms  as  a  checklist,  and  refer  to  them  from  time  to  time. 

9.  Be  non-judgmental.  Only  upon  completion  of  an  observation,  you  may  discuss  with  ramp  employee(s)  if  there  are 
any  questions  or  comments.  A  LOS  A  observation  is  not  an  i  nvestigation.  The  s  ole  purpose  of  post-observation 
discussion  i  s  t  o  gat  her  a  dditional  d  emographic  i  nformation  a  nd  hel  p  y  ou  better  understand  o  r  cl  arify  t  hings 
observed  earlier.  Do  not  ask  the  staff  why  they  did  something  incorrectly  or  did  not  do  something  at  all  (like  place 
a  chock).  This  is  n  ot  an  investigation,  and  such  questions  will  make  the  staff  feel  th  at  they  are  b  eing  judged  on 
their  performance. 

10.  Upon  completion  of  your  observation,  go  through  your  coding  and  notes  in  the  observation  forms,  and  write  up 
additional  notes  as  soon  as  you  can. 

Checklist  for  ramp  LOSA  observations 

Remember  to  prepare  the  following: 

□  Permission  to  observe  in  the  location  that  you  have  selected  (and  separate  permission  for  e.g.,  photographing) 

□  A  plan  of  what  and  how  you  are  going  to  observe,  which  people  you  should  talk  to 

□  An  open  mind  -  try  to  recognize  and  avoid  preconceptions 

□  Pen  and  notepad  with  LOSA  forms  and  supporting  documents  and  sufficient  paper  for  taking  notes 

□  Camera  or  audio  recorder  (if  they  are  a  part  of  the  LOSA  program  protocol  at  your  organization) 

□  Your  or  the  LOSA  program  manager’s  card  or  contact  information  so  the  people  you  have  observed  and  talked  to  can 

get  in  touch  with  you  if  they  have  questions  or  further  information 

□  Personal  safety  protection  gear  (e.g.,  earplugs,  goggles,  hard  toe  shoes,  reflective  safety  vest,  knee  pads) 

□  Ramp  employee  tools  (e.g.,  radio) 

Remember  to  do  the  following: 

□  Arrive  in  time  and  be  well  prepared 

□  Introduce  yourself  and  explain  why  you  are  there 

□  Ask  permission  to  do  the  LOSA  observation 

□  Explain  briefly  how  information  collected  will  be  used  and  toward  what  goal 
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□  Do  not  disturb  the  people  you  are  observing 

□  Look  and  listen  closely  and  take  notes 

□  When  it  is  possible,  complement  your  observations  with  a  short  discussion  with  the  employee  but  only  upon  completion 

of  your  observation 

□  Ensure  that  the  information  will  be  kept  anonymous  and  safe 

□  Write  up  your  notes  and  finish  up  coding  right  after  each  observation 

□  Make  sure  that  all  materials  collected  are  labelled  properly 

□  Dress  properly  and  wear  all  required  safety  and  security  badges  visibly 

Please  note  that  LOS  A  program  does  not  replace  other  safety  programs.  It  is  unrealistic  to  think  that  a  LOS  A  program 
will  identify  all  threats  or  remove  all  risks  or  threats  identified  in  the  LOSA  observations.  For  example,  suppose  a  ramp 
LOSA  observer  observed  an  employee  wearing  inappropriate  shoes,  but  according  to  the  LOSA  protocol,  did  not 
intervene.  This  employee  continued  wearing  inappropriate  shoes  during  and  after  the  LOSA  observation.  This  employee 
might  get  injured  later,  which  posed  a  risk  for  the  company.  An  organization  needs  to  recognize  what  are  acceptable  risks 
that  might  be  identified  in  the  LOSA  observations  but  would  not  be  eliminated  in  the  LOSA  program.  It  is  important  to 
connect  a  LOSA  program  with  other  safety  programs  (e.g.,  ASAP),  and  establish  a  means  to  handle  risks  or  threats 
identified  in  the  LOSA  observations  in  a  non-punitive  way. 

Please  refer  to  Appendix  C  for  written  protocols  for  LOSA  observers.  Appendix  D  lists  some  frequently  asked 
questions  about  M-LOSA  and  R-LOSA  programs. 


TIPS: 

A-  Observers  need  to  review  any  written  standard  procedures  for  a  given  task  to  be  observed. 

A-  Take  the  “fly-on-the-wall”  approach. 

A-  Never  intervene  unless  there  is  an  imminent  danger  to  the  employee(s),  the  aircraft,  or  the  equipment. 

A-  To  ensure  the  quality  of  data,  never  speculate  about  the  threats  or  other  information  during  your  observation 
unless  you  see,  hear,  smell,  or  touch  to  confirm  them. 

A-  Use  the  LOSA  observation  forms  as  a  memory  aid  (i.e.,  checklist)  and  refer  to  them  from  time  to  time. 

KEY  REMINDERS: 

A-  Expect  a  steep  learning  curve,  and  allow  sufficient  time  for  the  observers  to  practice  and  calibrate. 

FOR  ADDITIONAL  READING: 

A-  Field  observation  materials  and  templates  are  available  at  the  “Training”  section  of  the  project  website: 
www.MRLOSA.com 
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Steps  8-9:  Validate  LOSA  Data  &  Populate  Database 

Before  analyzing  data,  it  is  critical  to  first  verify  it  by  convening  a  meeting  of  local  experts.  There  are  several  steps  to 
ensure  that  LOSA  data  are  consistently  and  accurately  recorded  before  meaningful  analyses  can  be  conducted: 

•  Observers  are  trained,  calibrated,  tested,  and  recalibrated; 

•  Objective  observation,  not  subjective  evaluation,  is  the  basis  of  the  observation; 

•  The  written  description  and  comments  are  emphasized  as  key  to  high-quality  data; 

•  The  observers  and/or  analysts  apply  consistent  coding  to  the  observations;  and 

•  The  data  verification  group  checks  the  analyst’s  coding  against  specific  procedures. 


Figure  13.  Steps  8-10:  Validate  Data,  Populate  and  Maintain  Database,  &  Compile  a  Report 

LOSA  data  reveal  strengths  and  vulnerabilities  in  an  organization’s  operations.  The  data  analyst  should  investigate  the 
prevalence  and  management  of  different  threats  and  errors.  Although  certain  types  of  comparisons  seem  obvious,  many 
analyses  can  and  should  be  based  upon  hunches  and  theories  derived  from  local  knowledge  of  operations.  If  the  analyst 
knows  how  fleets  and  operations  are  managed,  comparisons  that  reflect  this  structure  can  be  made.  If  the  analyst  knows 
the  kinds  of  information  that  might  be  useful  to  training,  safety,  or  to  domestic  or  international  flight  operations,  results 
can  be  tailored  to  these  particular  aspects  of  the  operation. 

Feedback  from  various  stakeholders  is  critical  during  this  stage  of  data  analysis.  The  analyst  should  not  hesitate  to 
distribute  early  drafts  to  key  people  within  the  organization  who  are  familiar  with  LOSA  to  cross-verify  the  results.  This 
helps  validate  derived  trends  and  gives  other  personnel  ownership  of  information  derived  from  the  data. 

Patterns  emerge  as  the  data  are  analyzed.  Certain  errors  occur  more  frequently  than  others,  certain  airports  or  threats 
emerge  as  more  problematic  than  others,  certain  SOPs  are  routinely  ignored  or  modified,  and  certain  maneuvers  pose 
greater  difficulty  for  adherence  than  others.  These  events  and  practices  form  the  basis  of  suggested  targets  for 
enhancement. 

Step  10:  Compile  a  LOSA  Report 

The  next  stage  in  the  LOSA  implementation  process  is  a  written  report  that  presents  the  overall  findings  of  a  given 
round  of  LOSA  observations.  With  a  large  amount  of  observational  data  accumulated,  it  is  easy  to  fall  into  the  trap  of 
trying  to  present  everything.  The  report  should  be  concise  and  present  only  the  most  significant  trends  from  the  data. 

Along  with  the  results,  the  report  should  provide  an  initial  list  of  targets  for  enhancement.  Targets  need  to  be  data-driven 
and  action-focused.  Some  example  targets  that  might  emerge  from  a  LOSA  include:  Understand  automation  errors  on  the 
new  fleet,  investigate  conditions  at  airports/stations  X  and  Y,  improve  management  of  adverse  weather  threats,  investigate 
high  rate  of  Minimum  Equipment  List  (MEL)  items  on  the  ABC  fleet,  and  develop  a  module  on  intentional 
noncompliance  errors  for  refresher  training. 

Generating  reports  with  the  LOSA  database  can  be  accomplished  using  Microsoft  SQL  Server  Reporting  Services  or  a 
similar  reporting  utility.  Such  services  can  be  provided  by  a  Business  Analyst  with  Microsoft  SQL  Server,  Microsoft 
Access,  and  reporting  experience.  Though  not  required,  it  is  beneficial  if  business  analysts  for  LOSA  have  a  basic 
understanding  of  the  aviation  maintenance  or  ramp  environments.  If  you  wish  to  incorporate  LOSA  information  with  other 
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reporting  tools  available  in  your  company,  you  should  obtain  the  services  of  database  administrators  and  business  analysts 
familiar  with  your  current  systems. 

The  LOS  A  database  tool  provides  a  collection  of  predetermined  (“canned”)  reports.  The  canned  reports  are  limited  in 
number  and  focus.  They  are  generated  by  automatically  extracting  the  data  from  the  LOS  A  database  and  sending  it  to 
Microsoft  Excel  Pivot  Tables.  These  reports  can  then  be  modified  with  any  of  the  tools  available  in  Microsoft  Excel.  The 
canned  reports  consist  of: 

1 .  All  Cross-Reference  Reports:  All  data  from  the  database  in  one  spreadsheet. 

2.  Threat  Code  -  Totals  Matrix:  All  reported  threat  codes  and  the  frequency  that  they  were  reported. 

3.  Threat  Codes  by  Effectively  Managed:  All  reported  threat  codes  and  how  they  were  managed. 

4.  Error  Outcomes:  All  reported  errors  and  whether  the  result  was  Inconsequential,  Undesired  State,  or  Additional 
Error. 

5.  Error  Outcome  by  Observation  Type:  Error  outcomes  reported  for  each  form  (e.g.,  Install,  Complete  Restore). 

6.  Threat  Codes  by  Error  Outcomes:  All  reported  threat  codes  and  the  associated  error  outcome  (i.e., 
Inconsequential,  Undesired  State,  or  Additional  Error). 


Using  the  LOSA  data 

A  well-conducted  and  well-analyzed  round  of  LOSA  observations  identifies  strengths  and  vulnerabilities  in  an 
organization’s  operations.  It  provides  this  information  in  a  quantifiable  form  against  which  targets  can  be  specified  and 
improvements  can  be  measured.  The  following  example  briefly  illustrates  the  step-by-step  integration  of  LOSA  data  into 
the  safety  change  process: 

(1)  An  airline’s  LOSA  results  indicated  that  two  cowlings  a  month  were  being  damaged  because  of  leading  edge  device 
activation.  Because  observations  were  scheduled  across  the  operations,  and  the  number  of  observations  exceeded  50,  the 
LOSA  implementation  team  was  confident  that  the  percentage  is  an  accurate  representation  of  operations  as  a  whole. 

(2)  Following  management  briefings  and  extensive  discussion,  a  target  for  improvement  was  created  to  reduce  cowling 
damage  incidents. 

(3)  An  action  committee  was  formed  to  conduct  a  focused  LOSA  to  determine  the  cause  of  the  damage.  The  focused 
LOSA  revealed  that  the  lock-out  tag-out  procedures  were  being  incorrectly  executed.  A  contributing  factor  was  lengthy 
procedures.  The  committee  found  that  it  was  possible  to  lock  out  and  tag  out  the  leading  edge  devices  safely  with  fewer 
steps.  The  new  procedure  was  implemented. 

(4)  A  LOSA  was  repeated  6  months  after  the  first  LOSA.  The  data,  once  aggregated  and  analyzed,  showed  cowling 
damage  incidents  from  leading  edge  device  activation  had  been  reduced  by  over  90%. 

Data  from  a  particular  LOSA  program  can  be  cross-referenced  with  data  from  the  ASAP  and/or  QA  audit  programs. 
That  would  depend  on  the  sophistication  of  an  organization’s  SMS  and  the  extent  to  which  various  safety  programs  within 
the  organization  can  cooperate.  Each  data  source  provides  unique  yet  complementary  evidence  of  the  company’s  safety 
status.  In  the  above  example,  the  organization  might  track  cowling  damages  through  its  QA  audit  program.  New 
maintenance  procedures  could  then  be  implemented  into  procedures  and  training.  To  see  if  technicians  are  incurring 
problems  with  the  new  procedure,  the  QA  data  can  be  cross-referenced  with  ASAP  reports  of  events  resulting  from  lock¬ 
out  tag-out  procedures.  This  way,  the  organization  does  not  have  to  wait  until  the  next  LOSA  to  learn  if  its  interventions 
are  successful. 

LOSA  data  are  useful  in  another  way.  LOSA  presents  a  broad  view  of  operations.  A  repeat  LOSA  can  maintain  that 
broad  focus.  For  example,  did  the  changes  that  were  introduced  after  the  first  LOSA  improve  results  in  one  area,  only  to 
cause  problems  in  another?  Checklist  adherence  may  have  improved,  but  did  error  detection,  the  superordinate  goal  of 
improving  checklist  adherence,  actually  improve?  Or,  is  the  new  adherence  simply  cosmetic? 

In  summary,  the  data  collected  during  a  round  of  LOSA  observations  can  be  used  to  achieve  many  goals: 

a.  Identify  threats  in  the  organization ’s  operating  environment.  Observers  note  events  in  the  operational  environment 
(e.g.,  adverse  weather,  incomplete  documentation,  time  pressure,  and  how  they  are  managed  by  frontline  employees). 
High-prevalence  threats  and/or  threats  with  higher  mismanagement  rates  can  be  prioritized  for  further  investigation;  lower 
mismanagement  rates  signify  areas  of  strength.  For  example,  LOSA  can  promote  an  understanding  of  the  extent  to  which 
certain  procedures  pose  a  problem  for  frontline  employees.  LOSA  can  capture  the  strategies  that  frontline  employees  have 
adopted  to  deal  with  the  challenges.  Then  it  can  lead  an  organization  to  develop  special  procedures  or  advisories  to  help  its 
employees  manage  the  known  threat. 

b.  Identify  threats  from  within  the  maintenance  or  ramp  operations.  Observers  note  events  arising  from  within  the 
organization’s  own  operations  and  how  they  are  managed  (e.g.,  operational  time  pressure,  dispatch  errors,  aircraft 
malfunction/  MEL  items,  and  problems  with  interruptions,  other  personnel,  and  the  ramp).  A  high  number  of  threats 
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arising  from  dispatch  or  parts  might  signal  that  these  departments  require  attention.  It  could  reveal  challenges  with 
intergroup  cooperation  with  frontline  employees  or  that  procedures  are  inconsistent  across  departments.  As  above, 
prevalence  and  management  rates  provide  cues  for  prioritizing  action. 

c.  Assess  the  degree  of  transfer  of  training.  Data  provided  by  training  programs  can  provide  insight  on  whether 
training  concepts  are  learned  but  not  whether  they  are  actually  practiced.  A  LOSA  provides  that  operational  information. 
LOSA  can  be  reviewed  from  a  training  perspective  to  understand  which  areas  of  training,  if  any,  are  not  successfully 
transferred. 

d.  Check  the  quality  and  usability  of  procedures.  A  LOSA  provides  insights  about  potential  problems  with  procedures. 
For  example,  if  5%  of  observed  technicians  fail  to  properly  lock  out  and  tag  out,  there  may  be  a  problem  with  those 
technicians.  However,  if  50%  of  observed  technicians  make  the  same  error,  then  the  evidence  suggests  a  problem  with  the 
procedure.  Procedures  can  be  ill-timed,  overly-long,  confusing,  ambiguous,  and/or  compete  for  the  technicians’  attention 
with  other  more  important  activities.  A  LOSA  will  locate  problematic  procedures  and  policies  via  poor  adherence  rates.  A 
LOSA  can  also  identify  the  extent  of  procedural  deviations  across  fleets. 

e.  Identify  design  problems  in  the  human/machine  interface.  A  LOSA  captures  aircraft  design  and  reliability 
problems  on  different  fleets  that  can  highlight  systemic  flaws  in  design,  interface,  or  adaptation.  The  rate  at  which  certain 
errors  go  undetected  and  become  consequential  can  also  indicate  potential  design  vulnerabilities.  An  airline  can  feed  these 
LOSA  findings  back  to  the  aircraft  manufacturers,  as  well  as  use  findings  to  write  Standard  Operating  Procedures  and 
other  technical  documentation. 

f.  Understand  frontline  employees 9  shortcuts  and  workarounds.  With  experience  comes  expertise;  frontline 
employees  learn  ways  to  save  time  and  be  more  efficient.  These  techniques  are  rarely  seen  in  a  traditional  audit,  when 
performance  is  usually  done  by  the  book.  A  LOSA  provides  an  opportunity  for  the  organization  to  capture  collective 
expertise  from  within  the  frontline  employee  group,  and  then  share  that  information  with  all  its  employees  through  formal 
organization  communication  channels.  Using  LOSA,  false  expertise  —  the  adoption  of  a  shortcut  or  workaround  that  is 
flawed  in  its  safety  assumptions  —  can  also  be  identified  and  remedied.  Shortcuts  or  workarounds  indicate  that  frontline 
employees  were  often  aware  of  a  problem  in  existing  operational  procedures. 

g.  Assess  safety  margins.  Threats  and  errors  that  are  mismanaged  can  result  in  undesired  states,  if  sufficiently  serious. 

A  LOSA  provides  data  about  the  prevalence  and  management  of  these  incident  and  accident  precursors.  Thus,  an 
organization  acquires  data  about  how  closely  it  is  operating  to  the  edge  of  the  safety  envelope,  without  crossing  the 
boundary  into  an  incident  or  accident. 

h.  Provide  a  baseline  for  organizational  change.  LOSA  results  provide  baseline  and  outcome  measurement  data 
against  which  organizational  interventions  can  be  measured.  Using  a  medical  analogy,  this  is  akin  to  a  patient  deciding  to 
cut  out  fried  foods  upon  learning  of  a  high  cholesterol  count.  The  next  checkup  reveals,  in  quantifiable  form,  whether  this 
strategy  has  been  effective  in  reducing  cholesterol  or  whether  other  actions  are  necessary.  Similarly,  a  follow  up  LOSA 
provides  a  new  set  of  results  that  will  show  whether  the  organizational  changes  were  effective  in  reducing  certain  threats, 
errors,  and/or  undesired  states. 

i.  Provide  a  rationale  for  allocating  resources.  Because  LOSA  results  highlight  the  weaknesses  as  well  the  strengths  in 
an  organization,  the  results  provide  a  data-driven  rationale  for  prioritizing  and  allocating  scarce  organizational  resources 
toward  interventions. 


TIPS: 

Document  and  share  fact-based  data  to  illustrate  benefits  from  LOSA  efforts. 

A-  Use  the  information  —  create  action  plans,  implement  changes,  and  evaluate  results. 

KEY  REMINDERS: 

A-  Consistency  is  essential. 

FOR  ADDITIONAL  READING: 

A  Microsoft  Online  Reference  for  SQL  Server:  www.microsoft.com/sqlserver/en/us/default.aspx 
A-  SQL  Training  for  Beginners.  Online  Reference:  www.sqlcourse.com/index.html 

A-  FAA.  (2006).  Advisory  Circular:  Line  Operations  Safety  Audits  (AC  120-90):  Federal  Aviation  Administration. 
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Step  11:  Feedback  to  the  Employees 
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Figure  14.  Step  11:  Feedback  to  Employees 

It  is  critical  to  give  the  employees  feedback  on  all  LOSA  observations. 

First,  brief  the  management  and  labor  leadership  about  what  has  been  learned  and  action  items  derived  from  the  initial 
round  of  LOSA  observations.  The  LOSA  report  should  be  presented  to  management  in  maintenance  or  ramp  operations, 
training,  standards,  safety,  and  possibly  other  departments,  depending  on  the  results.  For  example,  representatives  from 
flight  operations,  dispatch,  and  cabin  may  want  to  hear  how  their  work  is  perceived  from  the  LOSA  observers’ 
perspective,  particularly  if  it  is  problematic.  A  briefing  to  the  union  or  employee  group,  as  applicable,  is  also 
recommended.  Once  the  various  departments  are  briefed  on  the  report,  they  will  likely  want  to  investigate  the  data  more 
deeply  themselves.  The  data  should  be  available  in  aggregate  form  for  them  to  review.  Some  of  the  more  detailed  LOSA 
comments  will  also  be  of  interest,  so  make  sure  to  de-identify  those  observations  before  distributing  aggregate  LOSA 
summary  and  comments. 

Second,  and  most  importantly,  brief  the  frontline  employees.  Technicians  and  ramp  personnel  should  also  be  informed 
of  the  significant  results  in  the  LOSA  report.  To  sustain  the  frontline  employees’  interest  in  the  LOSA  project,  make  an 
announcement  at  the  end  of  the  data  collection  phase  that  the  LOSA  observations  have  been  completed.  The  early  report 
can  simply  state  how  many  LOSA  observations  were  completed  and  on  which  fleets,  and  advise  the  workforce  when  to 
expect  the  results.  When  the  report  is  ready,  the  highlights  should  be  presented  to  the  frontline  employees,  either  as  one 
LOSA  debriefing  event,  spread  over  time  in  the  organization  newsletter,  or  other  safety  periodical,  or  other  channels  of 
communication,  such  as  safety  bulletin  boards  or  a  LOSA  website.  Frontline  employees  will  want  to  know  what  changes 
will  be  undertaken  as  a  result  of  the  LOSA,  and  they  should  be  a  part  of  the  change.  All  reporting  should  be  timely  and  not 
held  back  by  the  long  list  of  communications  and  legal  personnel,  who  are  often  insensitive  to  the  timeliness  of 
information  dissemination. 

Third,  after  the  management,  labor  leadership,  and  frontline  employees  are  informed  of  LOSA  findings,  it  is  critical  to 
continuously  monitor  the  safety  change  process  through  implementing  and  monitoring  actions  resulting  from  the  LOSA 
observations.  Historically,  organizational  safety  changes  within  aviation  organizations  have  been  driven  by 


43 


accident/incident  investigation  and  intuition.  Today,  organizations  must  deal  proactively  with  accident  and  incident 
precursors.  To  be  successful,  the  safety  change  process  must  be  data-driven. 

Measurement  of  daily  operations  is  fundamental  because  unless  an  organization  uses  systematic  measurement,  the 
perspective  it  has  on  the  strengths  and  weaknesses  of  its  operations  is  largely  based  on  anecdote  and  opinion.  A  LOS  A 
implementation  provides  specific  and  quantified  results.  To  take  full  advantage  of  this  specificity,  the  targets  for 
enhancement  that  arise  from  the  data  analysis  should  go  through  a  formal  safety  change  process  to  improve.  A  formal 
safety  change  process  provides  a  principled  approach  to  target  limited  resources  and  helps  the  organization  avoid  turf 
issues  by  clearly  defining  and  prioritizing  the  issues  that  impact  maintenance  or  ramp  operations.  The  basic  steps  of  a 
safety  change  process  include: 

1 .  Communicate  with  all  employees  at  every  step, 

2.  Measure  (with  LOS  A)  to  obtain  the  targets, 

3.  Perform  detailed  analysis  of  targeted  issues, 

4.  List  potential  changes  for  improvement, 

5.  Assess  risk  and  prioritize  changes, 

6.  Select  and  fund  changes, 

7.  Implement  changes, 

8.  Allow  time  for  changes  to  stabilize,  and 

9.  Re-measure. 

Once  your  organization  has  done  an  initial  LOS  A,  a  critical  question  is  when  to  conduct  the  next  one.  LOS  A  data 
provide  a  baseline  against  which  to  measure  improvements.  A  realistic  timeframe  to  review  LOS  A  results,  develop  and 
implement  action  plans,  and  monitor  results  is  six  months  to  one  year.  Hence,  to  measure  the  effectiveness  of 
organizational  changes,  a  repeat  LOS  A  every  six  months  to  one  year  is  recommended. 


TIPS: 

4-  Communicate  the  LOS  A  results  in  a  timely  manner. 

A-  Use  the  information  —  create  action  plans,  implement  changes,  and  evaluate  results. 

KEY  REMINDERS: 

A-  Consistency  is  essential. 

FOR  ADDITIONAL  READING: 

A-  Strategic  Employee  Communications:  Solutions  that  Influence  Behaviors  and  Attitudes.  Online  Reference: 
www.ketchumperspectives.com/archives/2004_i2/intemal/strategic.php 
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V.  Continuous  LOSA  Application  Phase 
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Figure  15.  Steps  12  &  13  :  Continue  LOSA  Effort  and  Conduct  Industry  Infoshare 

Step  12a:  Integrate  LOSA  Data  and  Data  From  Other  Safety  Programs 

The  TEM  model  applies  at  different  levels  and  sectors  within  an  organization  and  across  different  organizations  within 
the  aviation  industry.  For  example,  to  enhance  aviation  safety  and  efficiency,  International  Civil  Aviation  Organization 
(ICAO)  (2005)  introduced  the  TEM  framework  and  TEM-based  tool  called  Normal  Operations  Safety  Survey  (NOSS)  to 
the  Air  Traffic  Services  community,  and  the  Air  Traffic  Control  (ATC)  community  in  particular.  According  to  ICAO 
(2005),  NOSS  is  intended  to  furnish  the  ATC  community  with  a  means  for  providing  robust  data  on  threats,  errors,  and 
undesired  states.  NOSS  data  and  safety  data  from  conventional  sources  should  focus  the  safety  change  process  on  the  most 
critical  areas.  The  TEM  framework  helps  make  operational  personnel’s  implicit  awareness  of  the  threats  explicit  and 
principled,  and  therefore  manageable.  Dispatch  Operations  Safety  Audit  is  an  adaptation  of  the  LOSA  methodology  and 
the  TEM  model  to  airline  flight  dispatch  operations  (FAA,  2006).  Transfer  of  LOSA  methodology  to  other  areas 
eventually  helps  provide  a  360-degree  perspective  on  interactions  between  various  aircraft  entities  (FAA,  2006).  These 
previous  transfers  of  LOSA  echo  the  importance  and  timeliness  of  extending  LOSA  effort  into  aviation  maintenance  and 
ramp  operations. 


Complement  existing  safety  programs  in  SMS 

Safety  reporting  is  one  element  of  SMS  SRM  implementation.  The  SRM  function  provides  for  initial  identification  of 
hazards  and  assessment  of  the  hazards  for  risk.  Organizational  risk  controls  are  developed,  and  once  they  are  determined 
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to  be  capable  of  bringing  the  risk  to  an  acceptable  level,  they  are  employed  operationally.  Consequently,  employee 
reporting  processes  and  systems  are  an  essential  ingredient  in  identifying  known  safety  hazards  in  an  organization  to  fulfill 
the  SMS  SRM  requirement. 

Robust  SMS  employee  reporting  programs  will  have  multiple  layers  of  employee  safety  reporting  processes  that  may 
include  Event  Reporting,  ASAP,  and  LOSA  programs.  While  identifying  every  conceivable  hazard  would  be  unlikely, 
aviation  service  providers  are  expected  to  exercise  due  diligence  in  identifying  significant  and  reasonable  foreseeable 
hazards  related  to  their  operations.  The  SMS  framework  specifies  that  the  aviation  service  provider  must  provide  for  a 
means  of  employee  communication  that  allows  timely  submission  of  reports  on  safety  deficiencies  and  clear  lines  of 
communication  both  up  and  down  the  organizational  chain  regarding  safety  matters.  Employees  must  be  encouraged  by 
top  management  to  use  the  employee  reporting  system  without  fear  of  reprisals.  Data  from  the  safety  reporting  and 
feedback  system  (LOSA,  ASAP,  Event  Reporting)  should  be  monitored  to  identify  emerging  hazards.  Despite  the  program 
differences,  data  from  one  program  can  be  cross-referenced  and  used  to  guide  data  collection  in  another.  For  example, 
ASAP  reports  may  highlight  a  problem  with  lock-out  and  tag-out  (LOTO)  procedures.  This  information  can  be  given  to 
the  LOSA  implementation  team,  which  can  then  target  more  observations  on  LOTO  to  understand  the  magnitude  and 
specifics  of  the  problem. 

As  a  part  of  SMS,  the  organization  must  continuously  measure  the  effectiveness  of  the  SMS  and  of  safety  risk  controls 
through  the  use  of  the  safety  and  quality  policies,  objectives,  audit  and  evaluation  results,  analysis  of  data,  corrective  and 
preventive  actions,  and  management  reviews.  LOSA  is  one  of  the  tools  that  allow  organizations  to  meet  this  requirement. 
Effective  LOSA  data  mining  and  trending  provide  the  means  for  assessing  root  cause  analysis,  and  taking  effective, 
mitigating  actions  with  changes  to  policies,  procedures,  equipment,  and  tooling.  Consequently,  it  is  essential  with  any 
LOSA  program  integration  that  an  electronic  data  collection  system  be  deployed  that  allows  for  an  effective  and  timely 
data  analysis  process.  This  is  a  key  ingredient  with  any  of  the  employee  reporting  systems  that  are  ultimately  integrated 
into  the  SMS. 

LOSA  integration  into  an  organization  should  be  considered  only  as  one  of  several  key  employee  reporting  programs 
that  provides  a  continual  data  stream  of  frontline  operational  data  to  the  airline.  LOSA  data,  coupled  with  data  from  other 
employee  reporting  programs  such  as  ASAP  and  Event  Reporting,  can  be  a  powerful  tool  in  defining  operational  hazards 
and  allowing  for  timely  risk  mitigating  actions  that  can  lower  risks  and  improve  the  overall  safety  environment  of  the 
airline. 


Step  12b:  Analyze  Return  on  Investment  (ROI) 

LOSA  can  help  ensure  continuing  safety,  reduce  operating  costs,  and  make  a  significant  contribution  to  identifying 
predictive  indicators  for  the  overall  safety  management  system.  These  statements  seem  obvious.  However,  it  is  best  when 
such  intuitive  statements  are  supported  with  data.  It  is  possible  and  relatively  straightforward  to  measure  the  effect  of  the 
LOSA  and  resulting  interventions.  It  takes  organizational  commitment  to  use  existing  data  to  show  the  impacts  on 
financial  and  safety  bottom  lines. 

The  term,  ROI  is  rooted  in  finance.  In  the  simplest  sense,  it  is  a  basic  math  problem  of  comparing  the  amount  invested 
to  the  amount  returned  (i.e.,  Return  over  Investment)  during  a  specified  time  period.  Of  course,  you  must  subtract  the 
amount  invested  from  the  return  before  you  divide  the  return  by  the  investment.  Figure  16  shows  the  basic  ROI  model. 
This  section  is  not  meant  to  be  a  treatise  on  ROI  but  merely  makes  the  point  that  such  calculations  are  feasible. 


Figure  16.  The  Basic  Return  on  Investment  Model 


The  scenario  below  is  a  fictitious  example  of  an  ROI  calculation  of  an  issue  discovered  by  LOSA.  Because  it  was  not 
fully  proven,  it  was  necessary  to  estimate  that  there  was  a  0.6  probability  that  the  intervention  identified  by  LOSA  would 
actually  work.  The  calculation  is  based  on  that  estimate. 


46 


In  one  month,  an  airline  encountered  over  100  additional  30-minute  gate  delays  because  technicians  could  not  process 
accurate  maintenance  documentation  in  a  timely  manner. 

A  set  of  targeted  LOSA  observations  showed: 

•  Line  maintaining  technicians  spent  excessive  time  citing  all  the  maintenance  manuals  and  appropriate  job  cards  in 
the  retum-to-service  paperwork. 

•  Flight  crews  often  rejected  packages  because  they  “seemed  incomplete.” 

•  Airworthiness  and  safety  were  never  in  question,  but  regulatory  compliance  was  often  the  issue  causing  a  delay. 

•  Flight-line  technicians  complained  that  they  often  were  too  far  away  from  documentation  terminals  and  printers. 

The  estimated  ROI  was  calculated  in  the  following  manner: 

•  Manufacturer’s  estimate  was  $6,000  per  hour  of  gate  delay  for  narrow-body  aircraft. 

•  One  hundred  30-minute  delays  cost  $3.6M  per  year  (600  hours  of  delay/year  x  $6, 000/hour). 

•  Maintenance  analysts  determined  a  way  to  transmit  the  most  commonly  used  forms  and  job  cards  to  portable 
computer/printers  for  flight  lines  in  the  four  selected  hubs. 

•  The  cost  for  the  implementation  was  estimated  to  be  about  $500,000  per  hub  for  hardware  installation,  testing,  and 
implementation. 

•  The  airline  planned  to  invest  in  the  system  for  an  additional  $500,000  in  each  subsequent  year. 

•  The  analysts  estimated  that  these  new  systems  would  eliminate  about  60%  of  the  delays  (.6  probability  of  success). 
Table  6  shows  the  ROI  for  a  three-year  period. 


Table  6.  Three-Year  ROI  for  Airline  Documentation  Example 

First  Year 

((3,600,000  x.6)  -  $2,000,000)/  $2,000,000  =  ROI  =  8% 

Second  Year  ($500,000  added  for  support) 

((7,200,000  x  .6)  -  2,500,000)/$2,500,000  =  ROI  =  73% 

Third  Year  ($500,000  added  for  support) 

((10,800,000  x.6)  -  3,000,000)/  $3,000,000  =  ROI  =  116% 


The  example  calculations  de  monstrate  how  to  use  the  sim  pie  formula.  The  cal  culations  do  not  reapply  the  return  to 
subsequent  support  investment,  escalate  the  cost  of  error,  or  consider  the  time  value  of  money.  A  serious  economist  and 
team  of  maintenance  and  engineering  analysts  would  apply  appropriate  estimates  and  probabilities  to  each  intervention. 
However,  this  example  shows  that  even  simple  ROI  calculations  can  make  the  point. 

The  financial  calculations  are  more  straightforward  than  the  safety  calculations.  It  is  challenging  to  place  concrete  costs 
on  safety.  Therefore  “Safety”  is  less  conducive  to  ROI  calculation  than  the  many  interventions  that  may  save  money.  A 
reasonable  assumption  is  that,  as  LOSA  identifies  and  corrects  threats  in  an  organization,  it  also  affects  safety. 

ROI  cautions 

Start  slowly,  taking  small  steps.  Use  th e  LOSA  ROI  calculation  on  many  small  interventions.  Determine  the  validity 
and  reliability  of  your  estimates  on  the  small  projects  before  proceeding  to  the  large  proj  ects.  If  you  are  m  aking  ROI 
calculations,  in  advance  of  i  nterventions,  be  thoughtful  and  analytic  as  yo  u  estimate  the  probability  for  su  ccess.  Early 
success  on  the  small  proj  ects  will  build  not  only  your  predictive  accuracy  but  al  so  will  establish  c  orporate  trust  in  y  our 
ideas  for  new  interventions  to  ensure  safe  and  efficient  human  performance.  However,  please  note  accuracy  often  changes 
when  moving  from  the  simple  to  more  complex  projects  when  the  interactions  of  multiple  variables  are  encountered. 

FAA  tools  for  ROI 

The  FAA  offers  a  process  and  software  to  assist  with  ROI  calculations.  It  was  originally  designed  to  assess  the  ROI  on 
interventions  related  to  human  fatigue.  However,  the  model  (identical  to  the  description  above)  is  generic.  The  tool  is 
designed  for  mid-level  managers  rather  than  financial  analysts.  It  sets  th  e  foundation  to  assess  the  v  alue  of  the  many 
interventions  that  can  result  fro  m  ram  p  an  d  m  aintenance  LOSA  programs.  Th  e  software  is  av  ailable  at 
www.mxfatigue.com. 
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Alternatives  to  LOSA  ROI  calculations 

It  is  not  always  necessary  to  perform  calculations  to  justify  a  LOSA  program  or  the  process  and  product  improvement 
interventions  that  may  result  from  LOSA.  Organizations  have  plenty  of  Key  Performance  Indicators  (KPIs)  that  can 
demonstrate  the  impact  of  LOSA.  Since  KPIs  are  influenced  by  many  factors,  it  is  important  to  delimit  topics  to  be  sure 
that  a  KPI  change  is  really  attributable  to  a  LOSA. 

There  are  many  methods  to  show  the  value  of  LOSA.  Because  LOSA  identifies  specific  weaknesses  or  can  highlight 
excellent  processes,  it  is  possible  to  assess  impact.  LOSA  is  an  answer  to  the  adage  “If  you  can’t  measure  it  then  you  can’t 
improve  it.” 

Step  12c:  Success  Stories  and  Lessons  Learned 

It  is  critical  to  document  some  success  stories  and  lessons  learned  of  M-LOSA  and  R-LOSA.  At  times  it  can  be 
difficult  to  obtain  approvals  from  an  organization’s  Legal/PR/Corp  Communication  Departments  to  publish  and  distribute 
the  fact-based  examples.  As  discussed  in  Section  2.1,  senior  management’s  continuous  commitment  will  be  critical  to 
write  success  stories  using  fact-based  data. 

M-LOSA 

At  Airline  ABC,  M-LOSA  findings  help  to  make  deactivation  procedures  more  workable,  efficient,  and  safer.  As  an 
example,  767  leading  edge  device  deactivation  and  reactivation  procedures  took  three  hours  to  properly  tag  out  without 
individual  sign-offs.  An  M-LOSA  auditor  identified  this  inefficiency,  which  was  then  addressed  by  Tech  Publications  by 
rewriting  their  deactivation/reactivation  procedures.  Now,  with  individual  sign-offs,  this  modified  process  takes  between 
30  and  45  minutes  to  complete.  The  new  procedures  also  help  to  standardize  the  process  to  avoid  problems  caused  by  shift 
changes  (deactivation  and  reactivation  are  often  carried  out  on  different  shifts)  and  interruptions.  This 
deactivation/reactivation  procedure  has  been  implemented  in  the  entire  fleet.  Because  of  the  changes  implemented  by  M- 
LOSA,  the  threats  have  been  reduced  tremendously,  and  no  damage  to  the  aircraft  has  occurred  since. 

Reasons  the  lockout  and  tag  out  process  previously  took  so  long  involve  unnecessary  deactivation  of  some  systems. 
Lock-out  and  tag-out  of  the  leading  edge  devices  in  the  Aircraft  Maintenance  Manual  (AMM)  is  37  pages  long.  Some 
steps  required  multiple  section  reviews  between  different  sections  of  the  AMM.  The  new  workcard  that  emerged  from  M- 
LOSA  is  two  pages  long  with  different  steps  clearly  listed.  There  are  no  individual  sign-offs  following  the  AMM  (e.g., 
deactivate  the  slats  per  AMM  27-XXX). 

As  another  example,  a  previous  norm  was  that  there  was  no  huddle  (briefing)  before  the  pushback  of  ground 
equipment.  M-LOSA  identified  that  the  person  who  needs  to  take  responsibility  for  the  huddle  is  the  maintenance 
technician  in  the  cockpit. 

R-LOSA 

In  2008,  among  447  problems  identified  by  flight  ops  LOSA  at  Airline  ABC,  147  (29%)  were  ground  safety  issues.  An 
examination  of  flight  ops  LOSA  archival  data  revealed  that  the  industry  average  is  only  16%  for  flight  ops  ground  safety 
issues. 

To  improve  ground  safety  performance,  Airline  ABC  has  established  several  safety  programs  under  the  umbrella  of  its 
SMS;  for  example,  the  Safety  Recognition  Program.  Station  #1  has  the  same  ground  safety  programs  as  Station  #2  and 
added  the  R-LOSA  program.  Both  stations  improved  their  group  safety  performance  dramatically  over  a  three-year  span. 
Note  that  the  actual  data  for  the  third  year  are  only  available  from  January  through  October.  Monthly  averages  for  the  first 
10  months  were  used  to  estimate  November  and  December  ground  damage  mishaps,  and  consequently,  the  averages  for 
the  entire  year.  However,  the  improvement  observed  by  Station  #1  was  more  than  Station  #2,  which  can  be  potentially 
attributed  to  the  effectiveness  of  R-LOSA  program  (Note  that  Station  #l’s  initial  safety  performance  was  better  than 
Station  #2).  Ground  safety  performance  was  assessed  using  three  measures:  (1)  total  number  of  ground  damage 
occurrences;  (2)  ground  damage  mishap  rate  per  10,000  departures;  and  (3)  cost  of  ground  damage  mishaps. 

Ground  operation  mishaps  can  be  further  categorized  as  attributable  mishaps  and  non-attributable  mishaps.  Attributable 
mishaps  are  a  result  of  human  error  and  will  be  charged  back  to  the  responsible  department  or  vendor.  Non- Attributable 
Mishaps  include  FOD.  The  costs  are  not  recovered  for  these  mishaps.  Both  stations  showed  a  dramatic  decrease  in  the 
total  number  of  ground  damage  mishaps  over  the  three-year  span.  The  number  of  attributable  and  non-attributable  mishaps 
for  Station  #1  dropped  73%  and  85%,  respectively,  whereas  the  drops  for  Station  #2  were  58%  and  67%,  respectively.  The 
cost  of  ground  damage  also  decreased  at  both  stations  over  the  three  years. 

Per  every  10,000  departures,  Station  #l’s  attributable  ground  damage  mishaps  dropped  61%,  and  non-attributable 
mishaps  dropped  to  zero  over  the  three  years.  For  Station  #2,  both  attributable  and  non-attributable  mishap  rates  decreased 
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43%  and  45%,  respectively.  The  most  significant  improvements  were  observed  in  the  following  four  areas:  ground 
handlings  ops,  struck  by  vehicle  in  motion,  taxi-tow-push,  and  maintenance  operations. 


TIPS: 

A-  Calculate  ROI  to  measure  and  quantify  success. 

A-  Document,  communicate,  and  share  fact-based  data  to  illustrate  the  benefits  of  LOS  A  effort. 

A-  Document,  communicate,  and  share  lessons  learned. 

A-  Use  the  information — create  action  plans,  implement  changes,  and  evaluate  results. 

KEY  REMINDERS: 

A-  Consistency  is  essential. 

FOR  ADDITIONAL  READING: 
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A-  IATA  (2011).  Safety  Management  System  (SMS). 

www.iata.org/whatwedo/safety_security/safety/pages/safety_management_systems.aspx 
A-  Stolzer,  A.J.,  Halford,  C.D.,  &  Goglia,  J.J.  (201 1).  Implementing  Safety  Management  Systems  in  Aviation. 

Ashgate  Studies  in  Human  Factors  for  Flight  Operations.  Burlington,  VT:  Ashgate  Publishing  Company. 

A-  Johnson,  W.B.,  Sian,  I.B.,  &  Watson,  J.  (2000).  Measuring  the  Impact  of  Human  Factors  Interventions. 
www.hf.faa.gov/docs/508/docs/Johnson_ROI.pdf 

A-  Johnson,  W.B.  (2006).  Return  on  Investment  in  Human  Factors,  www.raes-hfg.com/reports/18oct06- 
investing/1 8oct06-Johnson.pdf 

A-  FAA  ROI  Calculator:  https://hfskyway.faa.gov/HFSkyway/FatigueMostRequested.aspx 
A-  Johnson,  W.  (201 1).  Collecting  “Predictive”  SMS  Data:  Line  Operations  Safety  Assessments  require  employee 
involvement  to  identify  safety  threats  to  1 1  areas  of  ramp  operations.  Ground  Support  Worldwide.  September 
2011.22-25. 

www.aviationpros.com/article/10343275/collecting-predictive-safety-management-systems-data 
A-  Return  on  Investment  Calculator:  www.money-zine.com/Calculators/Investment-Calculators/Return-on- 
Investment-Calculator/ 

A-  Demby,  G.  (2006).  How  to  Measure  Safety’s  ROI,  Part  3  of  3.  Safety  Economics. 

www.safetyxchange.org/fmancing-safety/how-to-measure-safetys-roi-part-3-of-3 
A-  SafetyXChange  (2008).  100  Performance-based  Criteria  Measuring  Safety. 

www. safetyxchange.org/tools/ 1  OO-performance-based-criteria-  for-measuring-safety 
A-  ASSE  White  Paper  Addressing  the  Return  on  Investment  for  Safety,  Health  and  Environmental  (SH&E) 
Management  Programs 

www.elcosh.org/en/document/63/d000047/asse-white-paper-addressing-the-return-on-investment-for- 

safety%252C-health-and-environmental-%2528sh%2526e%2529-management-programs.html 

*  THE  ROI  OF  SAFETY 

www.businessweek.com/adsections/2005/pdf/0534_roi.pdf 
A-  Flight  Safety  Foundation  (201 1).  Ground  Accident  Prevention  Cost  Model 

http://flightsafety.org/archives-and-resources/ground-accident-prevention-gap/ground-accident-prevention- 
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VI.  Critical  Success  Factors  for  a  LOSA  Implementation 

In  summary,  here  are  five  critical  success  factors  for  a  LOSA  implementation: 

(1)  Senior  Leadership  Support  and  Drive  for  Change 

Senior  management  commitment  is  vital  for  the  success  of  a  LOSA  program  and  key  to  initiating  the  drive  to  change 
the  organization  in  line  with  LOSA  results.  This  commitment  must  be  expressed  in  behavior  and  with  financial  support  to 
ensure  success  of  the  program.  Leaders  must  understand  the  process  and  its  purpose  to  be  able  to  communicate  the  value. 

(2)  Early  Stakeholder  Engagement 

Experience  has  shown  that  engaging  stakeholders  (including  those  who  are  likely  to  receive  LOSA  findings)  early  in 
the  LOSA  development  is  more  likely  to  lead  to  acceptance  of  LOSA  data  and  findings  and,  thus,  safety  culture  change. 
By  gaining  an  early  understanding  of  what  LOSA  is  and  how  it  works,  the  stakeholders  will  be  more  likely  develop 
ownership  for  the  program’s  development  and  success.  Stakeholders  may  include  representatives  from  various  functions 
such  as  training,  operations,  communication/marketing,  union/labor  groups,  the  regulators,  and  vendors. 

(3)  Communication  and  Promotions 

Promotion  and  awareness  training  prior  and  during  a  LOSA  implementation  is  essential  to  facilitate  awareness  and 
acceptance  by  the  general  workforce.  At  the  conclusion  of  an  assessment  period,  results  and  initiatives  of  the  LOSA  must 
be  communicated  to  the  operational  community  and  relevant  business  functions  such  as  the  training  department.  This  is  an 
important  step  as  it  demonstrates  the  organization’s  commitment  to  safety  change  to  the  frontline  employees,  thereby 
increasing  their  trust  in  the  LOSA  process  for  the  next  assessment. 

(4)  Non-jeopardy  Observations 

LOSA  implementation  must  follow  the  10  operating  characteristics  that  define  the  integrity  of  the  LOSA  process  and 
the  quality  of  LOSA  results  (FAA,  2006).  Ensuring  that  observations  are  not  used  for  disciplinary  or  performance 
management  is  critical  to  guarantee  the  LOSA  program  continues  to  witness  normal  behavior.  It  is  essential  (particularly 
in  the  implementation  phase  of  a  LOSA  program)  that  the  project  management  and  observation  team  have  the  respect  of 
their  peers  and  act  in  an  appropriate  manner  to  safeguard  the  reputation  of  the  program.  Programs  with  a  poor  reputation 
have  difficulty  gaining  acceptance  among  operators  and  management,  thereby  making  it  harder  to  initiate  change. 
Observers  should  be  selected  from  a  variety  of  stations  and  ranks  to  ensure  a  broad  cross-section  of  data  is  collected  from 
a  variety  of  perspectives  and  to  reduce  the  chance  of  normalization  in  the  data. 

(5)  Managing  Change  Process 

It  is  imperative  that  LOSA  data  are  used  appropriately  to  initiate  safety  change  within  the  organization  following  a 
round  of  assessment.  It  is  recommended  that  a  LOSA  Program: 

1 .  Obtains  a  baseline  measurement, 

2.  Provides  detailed  analysis  and  risk  assessment  of  identified  findings, 

3.  Identifies  and  assesses  potential  changes  to  mitigate  identified  risks, 

4.  Ensures  appropriate  actions  are  documented  and  implemented, 

5.  Allows  appropriate  time  for  actions  to  have  an  effect,  and 

6.  Measures  effectiveness  of  implemented  actions. 

VII.  Next  Steps 

This  guideline  should  be  used  to  lead  any  future  implementation  of  an  M-LOSA  or  R-LOSA.  A  few  complete  cycles  of 
implementation  processes  will  also  help  assess  the  contents,  structure,  and  language  of  this  guideline. 

Future  implementation  should  focus  on  evaluating  the  implementation  outcomes,  for  instance, 

•  identifying  commonalities  and  trends, 

•  assessing  risks  to  prioritize  corporate  safety  resources  and  effort, 

•  tracking  corrective  actions,  and 

•  monitoring  repetitive  findings  to  assess  the  effectiveness  of  prescribed  corrective  actions. 

Outcome  assessment  of  a  LOSA  implementation  will  help  generate  more  buy-in  of  the  program,  and  guide 
identification  of  leading  indicators  of  potential  (operational)  safety  issues,  service  issues,  and  personnel  injuries 
(occupational  safety  issues). 

The  M-LOSA  or  R-LOSA  program  is  expected  to  positively  leverage  peer  pressure  to  encourage  behavioral  change. 
The  periodic  assessments  can  help  evaluate  whether  identified  problems  have  been  resolved,  as  well  as  the  effectiveness 
of  safety  recommendations.  A  key  factor  for  the  implementation  team  is  to  set  the  frequency  and  scale  of  LOSA 
observations. 

As  more  organizations  implement  LOSA  programs,  an  industry-wide  LOSA  information  sharing  meeting  may  be  held 
biannually  to  exchange  best  practices  and  lessons  learned,  in  addition  to  zooming  in  on  fleet- wide  problems.  It  is  a  priority 
to  involve  more  airlines  in  the  M-LOSA  and/or  R-LOSA  initiative,  as  well  as  participants  from  the  regional  airlines  and 
Maintenance  Repair  &  Overhaul  (MROs)  communities. 
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Appendix  A.  Organizational  Change  Management  -  Stakeholder  Strategy 


STAKEHOLDER  PROFILE  NAME 

Profile  Description 

Defining  characteristics  of  this  group 

Significant  Changes  This  Group  Will  Experience 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

OVERALL  CHANGE  ASSESSMENT: 

□  HIGH  n  MEDIUM  □  LOW 

Involvement  Needs 

Communication  Needs 

Training/Education  Needs 

1. 

A. 

I. 

2. 

B. 

II. 

3. 

c. 

III. 

4. 

D. 

IV. 

5. 

E. 

V. 

6. 

F. 

VI. 

Key  Activities  with  Timeline  for  Involvement,  Communication,  Training/Education  Strategies 

Months 

1  2  3  4  5  6  7  8  9  10  11  12  13  14  15  16  17  18 

Involvement 

Communication 

Training 

Source:  Workshop  on  Project  Management  for  Supply  Chain  Managers.  Center  for  Supply  Chain  Management  Studies. 
Saint  Louis  University.  August  5  &  12,  201 1. 
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Appendix  B.  Resource  Requirements  - 
Responsible/Accountable/Consultted/Informed  Chart 


R-  Responsible  (for  doing  work)  A-  Accountable  (approving  authority) 

C-  Consulted  I-  Informed 


Person  or  Functional  Role 


Deliverable 
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Horizontal  Analysis 
If  You  Find: 


Analyzing  RACI  Chart 


Then  Ask 


No  R’s 

Is  job  getting  done?  Some  roles  may  be  waiting  to  approve,  be  consulted,  or 
informed.  No  one  sees  their  role  to  take  the  initiative. 

Too  many  R’s 

Is  this  a  sign  of  “over  the  wall”  activities?  “Just  get  it  of  my  desk  ASAP!” 

No  A’s 

Why  not?  There  must  be  an  “A.”  Accountability  should  be  pushed  down  to  the 
most  appropriate  level. 

Too  many  A’s 

Is  there  confusion?  “I  thought  you  had  it!”  It  also  creates  confusion  because  every 
person  with  an  “A”  has  a  different  view  of  how  it  is  or  should  be  done. 

Vertical  Analysis 


If  You  Find: 

Then  Ask 

Lots  of  R’s 

Can  or  need  the  individual(s)  stay  on  top  of  so  much?  Can  the  decision/activity 
be  broken  into  smaller,  more  manageable  functions? 

No  empty  spaces 

Does  the  individual(s)  need  to  be  involved  in  so  many  activities?  Are  they  a 
“gatekeeper”  or  could  management  by  exception  principles  be  used?  Can  C’s  be 
reduced  I’s,  or  left  to  the  individual’s  discretion  when  something  needs  particular 
attention? 

No  R’s  or  A’s 

Should  this  functional  role  be  eliminated?  Have  processes  changed  to  a  point 
where  resources  should  be  re-utilized? 

Too  many  A’s 

Does  a  proper  “segregation  of  duties”  exist?  Should  other  groups  be  accountable 
for  some  of  these  activities  to  ensure  checks  and  balances  and  accurate  decision 
making  throughout  the  process?  Is  this  a  “bottleneck”  in  the  process  -  is 
everyone  waiting  for  decisions  or  direction? 

Qualifications 

Does  the  type  or  degree  of  participation  fit  the  qualifications  of  this  role? 

Source:  Workshop  on  Project  Management  for  Supply  Chain  Managers.  Center  for  Supply  Chain  Management  Studies. 
Saint  Louis  University.  August  5  &  12,  201 1. 


56 


Appendix  C.  Written  Protocols  for  LOSA  Observers 

This  section  includes  some  written  protocols  you  can  use  when  you  deem  them  necessary. 

1.  Introduce  yourself 

A:  Hey,  man,  how  is  it  going?  I  am  Bob  Jones  from  Hangar  8.  My  role  today  here  is  a  LOSA  observer.  Have  you  heard 

of  LOSA  before? - [pause  for  response] 

[If  “Yes”] 

That’s  great,  man.  As  you  probably  recall  from  the  LOSA  awareness  training  you  attended  and  all  the  nice  looking 
posters  we  have  around  our  facility,  LOSA  is  a  peer-to-peer  observation  of  normal  operations.  As  the  observer,  my  goal  is 
to  be  a  “fly-on-the-wall”  and  step  back  to  allow  you  do  your  job  as  you  do  every  day.  Please  try  to  forget  I  am  even  here. 
Here  are  some  observation  forms  that  I  will  use  as  a  checklist  in  case  I  forget  something.  I  may  take  notes  from  time  to 
time.  But  let  me  assure  you  there  is  no  place  to  record  any  identifiable  personal  information  in  this  form,  and  I  was  trained 
to  not  write  down  any  identifiable  information.  Once  I  complete  the  observation,  I  may  ask  you  a  couple  of  questions 
regarding  some  demographics  or  things  to  help  clarify  questions  I  may  have  during  the  observations.  You  are  welcome  to 
ask  me  any  questions  too.  The  completed  observation  form  will  be  turned  into  the  LOSA  program  manager’s  office,  where 
they  will  be  entered  into  a  secured  database  and  further  analyzed.  We  can  expect  to  see  a  briefing  report  on  this  round  of 
LOSA  in  a  month.  The  report  does  not  contain  any  identifiable  information.  In  fact,  it  does  not  say  anything  about  any 
individual  observation,  and  instead  it  is  a  summary  of  findings  aggregating  all  observations. 

Do  you  have  any  questions  about  what  I  just  said?  Are  you  willing  to  take  part  in  LOSA? - [pause  for 

response] 

[If  “No”] 

Let  me  give  you  some  background  information  about  the  LOSA  program,  which  stands  for  Line  Operations  Safety 
Assessment.  LOSA  adopts  truly  proactive  and  predictive  strategy  to  address  aviation  safety.  As  a  voluntary  safety 
program,  LOSA  collects  safety  data  during  normal  airline  operations  and  was  originally  designed  for  flight  deck 
operations.  The  hazards  that  threaten  the  safety  of  flight  deck  operations  are  not  unique  to  that  environment.  Similar 
problems  are  present  during  maintenance  and  ramp  operations. 

As  you  probably  recall  from  the  LOSA  awareness  training  you  attended  and  all  the  nice  looking  posters  we  have 
around  our  facility,  LOSA  is  a  peer-to-peer  observation  of  normal  operations.  As  the  observer,  my  goal  is  to  be  a  “fly-on- 
the-wall”  and  step  back  to  allow  you  do  your  job  as  you  every  day.  Please  try  to  forget  I  am  even  here.  Here  are  some 
observation  forms  that  I  will  use  as  a  checklist  in  case  I  forget  something.  I  may  take  notes  from  time  to  time.  But  let  me 
ensure  you  there  is  no  place  to  record  any  identifiable  personal  information  in  this  form,  and  I  was  trained  to  not  write 
down  any  identifiable  information.  Once  I  complete  the  observation,  I  may  ask  you  a  couple  of  questions  regarding  some 
demographics  or  things  to  help  clarify  questions  I  may  have  during  the  observations.  You  are  welcome  to  ask  me  any 
questions  too.  The  completed  observation  form  will  be  turned  into  the  LOSA  program  manager’s  office,  where  they  will 
be  entered  into  a  secured  database  and  further  analyzed.  We  can  expect  to  see  a  briefing  report  on  this  round  of  LOSA  in  a 
month.  The  report  does  not  contain  any  identifiable  information,  in  fact,  it  does  not  say  anything  about  any  individual 
observation,  and  instead  it  is  a  summary  of  findings  aggregating  all  observations. 

Do  you  have  any  questions  about  what  I  just  said?  Are  you  willing  to  take  part  in  LOSA? - [pause  for 

response] 

2.  Answer  some  common  expected  questions  about  LOSA  program  and  observations 

2.a  Will  you  report  me  if  I  do  something  wrong?  Will  I  get  into  trouble? 

A:  No,  I  will  not  report  you  and  you  will  not  get  into  trouble,  let  me  assure  you  of  that.  The  purpose  of  the  LOSA 
observations  is  to  get  a  reading  of  our  normal  operations.  All  observations  are  of  a  non-jeopardy  nature,  and  that  all 
data  will  be  kept  strictly  confidential.  Here  is  a  letter  signed  by  the  highest  level  of  management  within  maintenance  or 
ground  operations  to  guarantee  that.  It  is  also  endorsed  by  our  employee  groups’  representatives. 

2.b  What  is  in  it  for  me? 

A:  A  LOSA  program  offers  many  benefits,  for  example,  identifying  strengths  and  weaknesses  of  normal  operations, 
reducing  undesirable  events  and  consequently  operating  costs,  and  improving  efficiency.  A  LOSA  program  is  very 
relevant  to  the  frontline  employees  like  you,  here  are  some  reasons  why: 

•  Assess  the  Degree  of  Transfer  of  Training.  A  LOSA  provides  a  consistent  measure  of  the  adequacy  and  focus  of 
training  programs  that  allow  an  operator  to  manage  those  programs  effectively. 
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•  Check  the  Quality  and  Usability  of  Procedures.  A  LOSA  provides  insights  about  potential  problems  with 
procedures.  A  LOSA  will  locate  problematic  procedures  and  policies  via  poor  adherence  rates.  A  LOSA  can  also 
identify  the  extent  of  procedural  deviations  across  tasks  or  fleets. 

•  Identify  Design  Problems  in  the  Human/Machine  Interface.  A  LOSA  captures  aircraft  design,  tooling  design,  and 
ground  service  equipment  design  issues  for  which  technicians  and  ground  servicing  personnel  have  developed 
“work  arounds.”  Some  of  these  interface  issues  may  be  able  to  be  fixed  so  that  no  work  arounds  are  required. 

•  Understand  Frontline  Employees  ’  Shortcuts  and  Workarounds.  With  experience  comes  expertise;  frontline 
employees  learn  ways  to  save  time  and  be  more  efficient.  These  techniques  are  rarely  seen  in  a  traditional  audit, 
when  performance  is  usually  done  “by  the  book.”  A  LOSA  provides  an  opportunity  for  the  organization  to  capture 
collective  expertise  from  within  the  frontline  employee  group,  and  then  share  that  information  with  all  its 
employees  through  formal  organization  communication  channels.  Using  LOSA,  false  expertise  —  the  adoption  of  a 
shortcut  or  work  around  that  is  flawed  in  its  safety  assumptions  —  can  also  be  identified  and  remedied. 

•  Provide  a  Rationale  for  Allocation  of  Resources.  Because  LOSA  results  highlight  both  the  strengths  and 
weaknesses  in  an  organization,  the  results  provide  a  data-driven  rationale  for  prioritizing  and  allocating  scarce 
organizational  resources  toward  continuous  improvement. 

2.c  What  are  you  going  to  do  with  the  data? 

I  will  submit  the  observational  data  I  gather  to  the  LOSA  program  manager.  A  safety  analyst  will  enter  the  data  into  a 
designated  database.  Please  note  that  the  LOSA  observational  data  housed  in  the  database  do  not  contain  any  identifying 
information  such  as  employee  name  or  employee  number.  LOSA  data  can  be  used  to 

•  Identify  threats  in  the  operating  environment. 

•  Identify  threats  from  within  the  maintenance  or  ramp  operations. 

•  Assess  the  degree  of  transference  of  training. 

•  Check  the  quality  and  usability  of  procedures. 

•  Identify  design  problems  in  the  human/machine  interface. 

•  Understand  frontline  employees’  shortcuts  and  workarounds. 

•  Assess  safety  margins. 

•  Provide  a  baseline  for  organizational  change. 

•  Provide  a  rationale  for  allocation  of  resources. 

2.d  Don’t  you  have  a  job  to  do  somewhere? 

A:  LOSA  is  a  volunteer  based  program.  I  volunteered  to  assist  our  organization  in  reducing  threats  and  errors  in  a  non- 
punitive  manner.  You  can  be  a  volunteer  LOSA  observer  too.  This  is  a  benefit  to  us  and  our  fellow  employees.  LOSA 
relies  upon  trust  and  open  communication  on  the  peer-to-peer  level.  My  training  and  observations  are  scheduled  during 
my  normal  work  shift. 

3.  Excuse  yourself  from  casual  conversation  with  the  technicians  being  observed 

A:  I  would  love  to  chat  more  about  that  game.  However,  right  now  I  can’t.  I  am  sorry.  I  need  to  be  a  “fly-on-the-wall.” 
To  allow  LOSA  capture  a  snapshot  of  true  normal  operations,  please  ignore  my  presence.  Let  us  catch  up  later. 

4.  Respond  to  technical  requests  for  help  from  the  technicians  being  observed 

A:  I  would  love  to  discuss  those  with  you.  However,  right  now  I  can’t.  I  am  sorry.  I  need  to  be  a  “fly-on-the-wall.”  To 
allow  LOSA  capture  a  snapshot  of  true  normal  operations,  please  ignore  my  presence.  I  will  make  sure  that  I  answer  your 
questions  right  after  I  complete  my  observation  here. 

5.  Respond  to  inquiries  from  management,  supervisor,  or  lead  who  may  be  walking  by 

A:  Hello,  I  am  Bob  Jones  from  Hanger  8. 1  am  a  LOSA  observer  conducting  observations  here  today.  I  checked  in  with 
the  management/supervisor/lead  this  morning.  Here  is  a  copy  of  letter  from  the  VP  of  operations  about  this  round  of 
LOSA  observations. 

6.  Intervene  immediately  in  what  the  technicians  are  doing  if  it  may  impact  the  individual’s  safety,  aircraft 
airworthiness  or  potential  damage  to  equipment 

A:  Sorry  I  in  terrupted  your  w  orkjustnow  b  ecause  I  saw  an  i  mminent  d  anger  to  yo  u,  (o  r/and  the  air  craft,  the 
equipment),  or  the  aircraft  was  going  to  be  left  in  a  non-airworthy  condition. 

Sorry  I  interrupted  your  work  just  now  because  I  noticed  you  were  unaware  of  this  error.  I  did  not  wait  for  you  to  finish 
a  task  and  then  have  you  backtrack  for  30  minutes  on  the  task  to  get  to  the  error. 
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7.  Conclude  the  observation  and  if  needed,  conduct  a  brief  discussion  with  the  technicians  being  observed 

A:  Thank  you  so  much  for  allowing  me  to  observe  you  today.  I  want  to  assure  you  that  your  identity  will  remain 
anonymous.  We,  LOSA  observers  were  trained  to  not  record  your  name  or  employee  number  on  the  LOSA  observation 
forms. 

May  I  ask  you  a  couple  of  questions?  You  don’t  need  to  answer  any  of  them  if  you  don’t  want  to  or  don’t  feel 
comfortable  about  sharing  the  information. 

Thank  you  for  clarifying  those.  Do  you  have  any  questions  for  me?  Here  is  my  (or/and  the  LOSA  program  manager’s) 
contact  information,  in  case  you  think  of  any  questions  and  you  need  to  get  hold  of  us. 
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Appendix  D.  Frequently  Asked  Questions  About  M-LOSA  and  R-LOSA  Programs 


General  Background 

1.  What  is  LOSA? 

A:  A  Line  Operations  Safety  Assessment  (LOSA)  is  an  observational  program  for  collecting  safety-related  data  during 
normal  operations.  It  is  a  means  for  a  company  to  self-assess  their  safety  margins.  Monitoring  routine  operations,  the 
cornerstone  of  the  LOSA  process,  identifies  at-risk  behaviors  so  that  they  can  be  proactively  managed.  The  process  also 
reinforces  positive  behaviors.  LOSA,  a  voluntary,  non-threatening,  non-punitive,  peer-to-peer  observational  process,  has 
been  implemented  in  airline  flight  operations  since  the  late  1990s.  Due  to  the  success  of  flight  LOSA,  the  process  has  been 
expanded  to  other  areas  of  the  aviation  operations,  such  as  air  traffic  control,  dispatch,  cabin  operations,  maintenance, 
customer  service,  and  ramp  operations.  On  the  flight  side,  LOSA  stands  for  “Line  Operations  Safety  Audit.”  To  better 
promote  voluntary  participation  and  a  proactive  safety  culture,  several  organizations  (e.g.,  International  Aviation 
Transport  Association)  now  use  LOSA  as  an  acronym  for  “Line  Operations  Safety  Assessment.” 

2.  Is  LOSA  just  another  audit  program?  We  already  have  similar  audit  programs  running! 

A:  LOSA  is  different  from  audit  programs  conducted  by  QA  Departments,  Safety  Departments,  or  external  agencies. 
One  essential  characteristic  of  LOSA  is  “peer-to-peer  observations”  of  normal  operations.  Maintenance  LOSA  and  Ramp 
LOSA  observations  are  not  carried  out  by  auditors  —  they  are  carried  out  by  peers.  There  are  two  advantages  of  having 
peers  conduct  the  observations.  First,  it  is  less  intrusive.  Let’s  face  it  —  it  is  natural  for  people  to  “work  to  the  rules”  when 
being  watched  by  an  auditor.  These  altered  behaviors  do  not  provide  an  accurate  reading  of  how  work  is  accomplished 
during  normal  operations.  On  the  other  hand,  frontline  employees,  while  they  may  alter  their  behaviors  at  the  start  of  a 
LOSA  program,  soon  ease  back  into  their  normal  behavior  when  peers  conduct  the  observation.  Second,  as  subject  matter 
experts,  who  are  familiar  with  local  operations  and  environment,  peer  observers  may  be  particularly  insightful  regarding 
where,  when,  and  what  to  look  for  during  an  observation.  LOSA  complements  other  safety  programs  (e.g.,  Aviation  Safety 
Action  Program,  Event  Reporting)  in  a  Safety  Management  System  as  a  predictive  hazard  identification  process. 

3.  What  are  threats?  What  are  errors? 

A:  Threat  and  Error  Management  (TEM)  provides  an  underlying  framework  for  LOSA  data  collection,  recognizing  that 
threats  and  errors  are  likely  to  occur  in  normal  operations. 

A  threat  is  any  condition  that  increases  complexity  of  the  operations  and  if  not  managed  properly  can  decrease  the 
safety  margin.  An  error  is  a  mistake  that  is  made  when  threats  are  mismanaged.  Errors  increase  the  probability  of  adverse 
operational  events  during  maintenance  or  during  ground  operations.  Errors  normally  occur  when  threats  are  mismanaged. 
However,  the  threat-error  linkage  is  not  necessarily  straightforward,  and  it  may  not  always  be  possible  to  observe  the 
threats  that  lead  to  an  error.  Error  outcomes  can  be  of  three  types:  inconsequential  (i.e.,  no  effect  on  safety),  an  undesired 
operational  state  (a  risky  or  unsafe  condition  for  the  aircraft,  equipment,  and/or  personnel),  or  additional  error(s)  linked 
together  across  time.  Managing  an  undesired  operational  state  can  be  considered  the  last  opportunity  to  avoid  an  unsafe 
outcome. 

4.  My  company  has  many  safety  programs.  Why  do  we  want  LOSA? 

A:  Managing  risks  associated  with  hazards  has  become  increasingly  important  in  modem  organizations,  especially  as 
the  aviation  industry  is  moving  toward  implementation  of  a  Safety  Management  System.  Three  methods  can  be  used  to 
find  hazards  that  can  be  assessed  for  risk: 

1 )  Reactive  hazard  identification  processes  are  investigations  of  accidents  and  incidents  to  determine  the  underlying 
factors  (hazards)  that  lead  to  the  accident/incident. 

2)  Proactive  hazard  identification  processes  look  for  at-risk  behaviors  (hazards)  before  they  lead  to 
accidents/incidents,  such  as  QA  audits  and  Hazard  Reporting  Systems. 

3 )  Predictive  hazard  identification  processes  also  look  for  at-risk  behaviors  (hazards)  before  they  lead  to 
accidents/incidents,  but  these  processes  involve  much  more  data  collection  than  proactive  processes.  This  allows 
the  prediction  of  accidents/incidents.  LOSA  is  a  primary  process  for  predictive  hazard  identification. 

During  a  LOSA  observation,  observers  record  and  code  identifiable  threats  and  errors  and  how  those  threats  and  errors 
are  managed.  The  data  from  LOSA  observations  provide  indicators  of  organizational  strengths  and  weaknesses,  which 
facilitate  the  development  of  countermeasures  to  operational  threats  and  errors. 
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Frontline  Employees 

5.  My  station’s  safety  performance  is  great.  If  it  ain’t  broken,  don’t  fix  it.  I  don’t  think  we  need  LOSA. 

A:  Current  great  safety  performance  does  not  mean  there  are  no  safety  risks  or  hazards  lurking  about.  Maybe  you 
have  just  been  lucky.  The  Federal  Aviation  Administration  (FAA)  will  soon  require  through  regulation  that  121  operators 
implement  a  Safety  Management  System  (SMS).  The  SMS  will  require  that  you  have  reactive,  proactive,  and  predictive 
hazard  identification  processes  in  place.  LOSA  will  meet  the  predictive  hazard  identification  process  requirement. 

6.  What  is  in  it  for  me? 

A:  A  LOSA  program  offers  many  benefits,  for  example,  identifying  strengths  and  weaknesses  of  normal  operations, 
reducing  undesirable  events  and  consequently  operating  costs,  and  improving  efficiency.  A  LOSA  program  is  very 
relevant  to  the  frontline  employees  like  you,  and  here  are  some  reasons  why: 

•  Assess  the  Degree  of  Transfer  of  Training.  A  LOSA  provides  a  consistent  measure  of  the  adequacy  and  focus  of 
training  programs  that  allow  an  operator  to  manage  those  programs  effectively. 

•  Check  the  Quality  and  Usability  of  Procedures.  A  LOSA  provides  insights  about  potential  problems  with 
procedures.  A  LOSA  will  locate  problematic  procedures  and  policies  via  poor  adherence  rates.  A  LOSA  can  also 
identify  the  extent  of  procedural  deviations  across  tasks  or  fleets. 

•  Identify  Design  Problems  in  the  Human/Machine  Interface.  A  LOSA  captures  aircraft  design,  tooling  design,  and 
ground  service  equipment  design  issues  for  which  technicians  and  ground  servicing  personnel  have  developed 
“work  arounds.”  Some  of  these  interface  issues  may  be  fixable  so  that  no  work  arounds  are  required. 

•  Understand  Frontline  Employees  ’  Shortcuts  and  Workarounds.  With  experience  comes  expertise;  frontline 
employees  learn  ways  to  save  time  and  be  more  efficient.  These  techniques  are  rarely  seen  in  a  traditional  audit, 
when  performance  is  usually  done  “by  the  book.”  A  LOSA  provides  an  opportunity  for  the  organization  to 
capture  collective  expertise  from  within  the  frontline  employee  group,  and  then  share  that  information  with  all  its 
employees  through  formal  organization  communication  channels.  Using  LOSA,  false  expertise  —  the  adoption 
of  a  shortcut  or  work  around  that  is  flawed  in  its  safety  assumptions  —  can  also  be  identified  and  remedied. 

•  Provide  a  Rationale  for  Allocation  of  Resources.  Because  LOSA  results  highlight  both  the  strengths  and 
weaknesses  in  an  organization,  the  results  provide  a  data-driven  rationale  for  prioritizing  and  allocating  scarce 
organizational  resources  toward  continuous  improvement. 

7.  Do  I  have  to  take  off  work  to  be  a  LOSA  observer? 

A:  No.  Your  training  and  observations  will  be  scheduled  during  your  normal  work  shift. 

8.  Will  LOSA  observers  report  me  to  the  company  if  I  do  something  wrong? 

A:  No.  LOSA  is  a  non-punitive  safety  program,  and  your  identity  will  remain  anonymous.  LOSA  observers  do  not 
record  your  name  or  employee  number  on  the  LOSA  observation  forms.  It  is  not  a  part  of  an  observer’s  task  to  report  your 
work  performance.  However,  a  LOSA  observer  is  trained  to  intervene  and  fix  a  problem  if  they  see  that  a  personal  injury, 
equipment  damage,  or  aircraft  damage  is  about  to  occur.  In  addition,  during  a  Maintenance  LOSA  observation,  the 
observer  may  see  that  a  task  has  been  done  incorrectly,  leading  to  an  airworthiness  issue.  In  this  situation,  the  observer 
needs  to  intervene  at  some  point  in  the  task  and  get  the  technician(s)  to  fix  the  problem,  so  that  the  aircraft  is  airworthy 
after  task  completion.  Even  in  these  situations,  employee  names  are  not  connected  to  the  observation.  Nobody’s  name  is 
ever  reported  to  management. 

9.  Do  I  have  to  participate  in  LOSA  program  as  an  observer? 

A:  No.  LOSA  is  a  volunteer  based  program.  Volunteering  to  assist  your  organization  in  reducing  threats  and  errors  in 
a  non-punitive  manner  is  a  benefit  to  you  and  your  fellow  employees.  LOSA  relies  upon  trust  and  open  communication  on 
the  peer-to-peer  level.  A  volunteer  has  the  personal  drive  and  motivation  toward  the  success  of  the  program  versus 
someone  who  is  told  they  must  engage. 

10.  Do  I  have  to  participate  in  LOSA  program  as  an  observed  worker? 

A:  LOSA  is  a  voluntary  program  in  which  people  agree  to  participate.  If  a  LOSA  observer  is  going  to  watch  you 
carrying  out  a  turn  on  the  ramp  or  observe  you  carrying  out  a  maintenance  task,  you  have  a  right  to  refuse  to  be  observed. 
However,  after  having  the  program  purpose  and  methodology  explained  to  you,  the  hope  is  that  you  will  be  happy  to  be 
observed.  Each  individual  carrier  or  ground  operator  is  fully  responsible  for  interpreting  and  defining  voluntary 
participation  to  its  frontline  employees  to  reach  a  mutual  understanding. 
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Manager/Lead/Supervisor 

11.  As  a  manager/lead/supervisor,  do  I  let  my  people  skip  work  to  conduct  LOS  A  observations? 

A:  Staff  do  not  “skip  work”  when  they  carry  out  a  LOSA  observation.  Management  supports  the  program,  and  the 
staff  volunteer  to  be  LOSA  observers  as  part  of  their  job.  The  real  issue  is  how  many  observations  can  be  carried  out 
and  still  make  sure  all  of  the  work  of  the  organization  gets  done.  Ideally,  with  senior  management’s  support,  there  will 
be  enough  man  power  to  allow  LOSA  observations  to  be  conducted.  The  number  of  LOSA  observations  that  can  be 
carried  out  during  a  given  time  period  will  be  dependent  on  workload  and  staffing.  The  frequency  can  be  one 
observation  per  shift,  one  observation  per  day,  or  two  to  three  observations  each  week  —  whatever  can  be 
accommodated.  Based  on  previous  successes,  LOSA  can  be  used  for  even  small  operations  where  the  trained 
observers  are  part  of  the  working  crew. 

12.  Does  LOSA  ask  my  employees  to  report  on  each  other? 

A:  No.  Employee  names  are  NEVER  recorded  on  the  LOSA  observation  forms.  LOSA  observers  report  on  the 
behaviors  of  employees  and  their  systemic  processes,  not  the  employees  themselves.  The  program  collects  data  and 
trends  of  these  behaviors  and  processes  without  punitive  consequences.  The  foundation  of  LOSA  is  that  it  is  peer-to- 
peer  and  non-punitive.  Since  peers  are  observing  peers,  with  a  non-punitive  understanding,  your  employees  will  have 
a  more  conducive  environment  for  an  observation  of  realistic  behavior  and  processes.  This  increases  the  opportunity 
for  the  observer  to  observe  the  associated  risks. 

13.  Will  LOSA  remove  all  threats? 

A:  No.  LOSA  is  not  a  “Silver  Bullet,”  but  it  does  provide  your  organization  the  benefit  of  data  collection  and 
trending.  This  data  will  help  identify  the  tacit  and  unknown  threats  to  your  system  and  processes.  LOSA  should  not  be 
viewed  as  a  standalone  process  but  a  complement  to  other  Safety  Management  System  processes  already  in  place. 
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Appendix  E.  A  List  of  Acronyms 


A4A  -  Airlines  for  America  (formerly  known  as  Air  Transport  Association  of  America) 

AC  -  Advisory  Circular 

AMM  -  Aircraft  Maintenance  Manual 

ASAP  -  Aviation  Safety  Action  Program 

ASRS  -  Aviation  Safety  Reporting  System 

ATC  -  Ai  Traffic  Control 

CASS  -  Continuing  Analysis  and  Surveillance  System 

CBT  -  Computer-based  Training 

FAA  -  Federal  Aviation  Administration 

FOD  -  Foreign  Object  Damage 

FOQA  -  Flight  Operations  Quality  Assurance 

GSE  -  Ground  Support  Equipment 

ICAO  -  International  Civil  Aviation  Organization 

LOTO  -  Lock-Out  and  Tag-Out 

LOS  A  -  Line  Operations  Safety  Assessment 

MCDBA  -  Microsoft  Certified  Database  Administrator 

MCSE  -  Microsoft  Certified  Systems  Engineer 

MCTS  -  Microsoft  Certified  Technology  Specialist 

MEDA  -  Maintenance  Error  Decision  Aid 

MEL  -  Minimum  Equipment  List 

M-LOSA  -  Maintenance  Line  Operations  Safety  Audit 

MRO  -  Maintenance,  Repair  &  Overhaul 

NOS S  -  Normal  Operations  Safety  Survey 

QA  -  Quality  Assurance 

RACI  -  Responsible/Accountable/Consulted/Informed 

REDA  -  Ramp  Error  Decision  Aid 

R-LOSA  -  Ramp  Line  Operations  Safety  Audit 

ROI  -  Return  on  Investment 

SA  -  Safety  Assurance 

SMS  -  Safety  Management  System 

SOP  -  Standard  Operating  Procedure 

SQL  -  Structured  Query  Language 

SRM  -  Safety  Risk  Management 

TEM  -  Threat  and  Error  Management 
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